Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.257052217s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:50:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.257100197s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...................................X.............X......X...] duration: 4.158797662s trace: https://jaeger.dev.storj.io/trace/5639d08444b9cd47 FAIL (2025-11-21 02:50:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.158977303s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [....................................X.............X......X..] duration: 4.37005372s trace: https://jaeger.dev.storj.io/trace/5639d08444b9cd47 FAIL (2025-11-21 02:50:16 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.370091201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.500459156s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:50:08 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.500508549s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.....................................X.............X......X.] duration: 4.289134686s trace: https://jaeger.dev.storj.io/trace/5639d08444b9cd47 FAIL (2025-11-21 02:49:19 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.28931052s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.594769447s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:49:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.594798842s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [......................................X.............X......X] duration: 5.002302758s trace: https://jaeger.dev.storj.io/trace/5639d08444b9cd47 FAIL (2025-11-21 02:49:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002327215s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................................X.............X......] duration: 4.393489213s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:48:43 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.393515232s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.394159722s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:48:35 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.394246257s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................................X.............X.....] duration: 4.355957883s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:47:45 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.35599379s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.316545001s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:47:36 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.31657614s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................................X.............X....] duration: 5.000868826s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:46:38 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000901067s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.754093207s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:46:32 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.754116411s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.80960984s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:45:58 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.809642424s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................X.............X...] duration: 5.000955816s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:45:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000979462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.305995734s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:45:04 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.306026732s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.119271413s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:44:48 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.119304484s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................................X.............X..] duration: 4.139898022s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:44:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.139919292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............................................X.............X.] duration: 4.006038831s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:44:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.006072635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.849101901s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:43:41 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.849200067s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............................................X.............X] duration: 5.001075562s trace: https://jaeger.dev.storj.io/trace/49b246d4fdffdc0e FAIL (2025-11-21 02:43:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001100971s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.251173005s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:43:03 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.251208061s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.962077721s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:42:51 UTC) orbiter warning: context deadline exceeded Duration: 3.962108449s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.670701047s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:42:40 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.670727296s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 4.01674628s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:42:20 UTC) orbiter warning: context deadline exceeded Duration: 4.016833116s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.240075843s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:41:59 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.240106724s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.695150371s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:41:37 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.695178796s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 4.801699936s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:41:11 UTC) orbiter warning: context deadline exceeded Duration: 4.801725786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.245109065s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:40:47 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.245139675s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 5.001267068s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:40:02 UTC) orbiter warning: context deadline exceeded Duration: 5.001493267s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.696486025s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:39:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.696501945s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.324219203s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:39:19 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.324313244s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 4.437406049s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:39:15 UTC) orbiter warning: context deadline exceeded Duration: 4.437435906s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 4.083455907s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:38:25 UTC) orbiter warning: context deadline exceeded Duration: 4.083487276s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.836645071s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:38:21 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.836677453s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.255706267s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:37:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.255733009s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 4.10729482s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:37:45 UTC) orbiter warning: context deadline exceeded Duration: 4.107325648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.618212077s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:37:23 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.618240201s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 4.061750486s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:36:48 UTC) orbiter warning: context deadline exceeded Duration: 4.061780232s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.841418881s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:36:16 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.841452715s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 5.001882087s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:35:50 UTC) orbiter warning: context deadline exceeded Duration: 5.001908938s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 4.90991127s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:35:21 UTC) orbiter warning: context deadline exceeded Duration: 4.909943232s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.102957856s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:35:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.102986872s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 4.050609792s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:34:22 UTC) orbiter warning: context deadline exceeded Duration: 4.050638858s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.34189313s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:34:09 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 5.341933645s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 4.251920121s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:33:59 UTC) orbiter warning: context deadline exceeded Duration: 4.251948895s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 4.140004164s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:33:39 UTC) orbiter warning: context deadline exceeded Duration: 4.140032699s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.26082873s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:33:17 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.260860652s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001011473s trace: https://jaeger.dev.storj.io/trace/5a70125f20f5565 FAIL (2025-11-21 02:33:01 UTC) orbiter warning: context deadline exceeded Duration: 5.001038894s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.143328701s trace: https://jaeger.dev.storj.io/trace/5860d9055b13d3d5 FAIL (2025-11-21 02:32:34 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 02:25:33.629924468 (282h21m32.181394939s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630349578 (282h21m32.181820039s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.630843158 (282h21m32.182313619s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.631501040 (282h21m32.182971502s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.632877661 (282h21m32.184348112s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.633957257 (282h21m32.185427718s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635008760 (282h21m32.186479221s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.635936546 (282h21m32.187407007s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:33.636636048 (282h21m32.188106509s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Uploading piece... Nov 21 02:25:34.461042315 (282h21m33.012512776s) [upload=248651]: Next returned nil segment Nov 21 02:25:34.461079235 (282h21m33.012549697s) [upload=248651]: Waiting for error group managing segments... Nov 21 02:25:34.850359259 (282h21m33.40182972s) [noise=true , address=185.167.204.206:4056 , piece_id=WP4BU4B4BJKASBDB7R7CB4IVBIP7NVRIHAOTZ2LWPG3XHMMC65TA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.852076179 (282h21m33.40354664s) [noise=true , address=185.167.204.206:4055 , piece_id=H54UI2MK43VZPELPTCU6XNLESYE6S5XZ4ZKAOV66TP3BBKKKDMFA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.856988012 (282h21m33.408458473s) [noise=true , address=185.167.204.205:4031 , piece_id=MCKJTCBSRINYPN27XNNFMXYPDHE3KVMSCWOB7VCOETNIO4F3PZVQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858141178 (282h21m33.409611639s) [noise=true , address=185.167.204.206:4067 , piece_id=TQD5JG4TGZYKCG4KOAWEUF6BSVY2IX35O2IB6VYKKH7ABNQEMRRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858405160 (282h21m33.409875621s) [noise=true , address=185.167.204.206:4041 , piece_id=S7UUW2R74ETDDNDVA73DRMGZ65YTPPLPWILZZ626SIG2YG55AOOA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.858612054 (282h21m33.410082515s) [noise=true , address=185.167.204.205:4023 , piece_id=NQMCNM7GAS2URY7JNN4DXMU2FZ7UAVHC42U55IVIGYUWXZ5DFOCQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859019680 (282h21m33.410490131s) [noise=true , address=185.167.204.205:4013 , piece_id=G5AFIVUYYIG3IAVQ2UN4MTL3NPXN326IOR77CXTU4BAIBFMICKXQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859464347 (282h21m33.410934808s) [noise=true , address=185.167.204.206:4044 , piece_id=6TMH2M7X2J6NUJ7H7YQ44MK3CHAHTONHFVB767OLQ7LVWYS6SWSQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.859565901 (282h21m33.411036352s) [noise=true , address=185.167.204.206:4071 , piece_id=CWYOC5U5SIHG24FJA76NVMYJG47L55GDKXTZ7FHENXXGL6NNLNCA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866136853 (282h21m33.417607324s) [noise=true , address=185.167.204.205:4027 , piece_id=DTF2OMEXXT2XFQB2RQOKL66K4AOIC3DTZOTE2ZGIMYWVPCTKJWJA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.866678734 (282h21m33.418149195s) [noise=true , address=185.167.204.206:4043 , piece_id=OBBQPYJ5QWYI3IDE7ROOB62NCIA425S2RW6N4BRRNXDSN7XTH3VA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867348910 (282h21m33.418819361s) [noise=true , address=185.167.204.205:4015 , piece_id=UC43M7E3DTT7P6X5FZR63SXMW3B7Y7LNX3CIXI5YGWVIPVXHRWWQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.867982197 (282h21m33.419452668s) [noise=true , address=185.167.204.206:4058 , piece_id=WJFVIB3443AETF6JXCW2CAYSL2E5SZWD3XITVJICPRBJUU7BO2OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.868482399 (282h21m33.41995286s) [noise=true , address=185.167.204.206:4037 , piece_id=ZTVJJC7I2VQ3ZO62Y2SNZKV64WYLEXHL2CJNDORXID6NBWWZSOYQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:34.880010110 (282h21m33.431480601s) [noise=true , address=185.167.204.205:4025 , piece_id=ZWXX3V2QD4QSVDNMMDJ5GSSLIMSAK5IZOHOEB2SI3TFMPUXRRVXA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.419467556 (282h21m33.970938017s) [noise=true , address=81.171.14.70:4044 , piece_id=WAQ3SBHX6TRISFEOPE4Y6O2HSE2NYVC257V6ROMZSFVJO7YLCGRQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.424133200 (282h21m33.975603661s) [noise=true , address=81.171.14.70:4038 , piece_id=X67ZAHX3AIYBBWES2PL5TB6FKR3DGZBALSLTITODERBWUI2OILVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.433560922 (282h21m33.985031383s) [noise=true , address=81.171.14.70:4041 , piece_id=AAUJYA6S7AWTCUO33QBNLCFUKRLGY6NLGFHRECJ5I63H53J46SMQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.452011215 (282h21m34.003481676s) [noise=true , address=81.171.14.70:4047 , piece_id=ZNKZHAJAP24ZF6ZBV6762YY7LCRUSQCCL5E53WU5TMZ457CRZ3RA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.508011640 (282h21m34.059482111s) [noise=true , address=81.171.14.70:4035 , piece_id=VI2R7YZTWKO4NNTYCTKDPAMHAA2CA5UA54ZJJOQXJ5FG6YY4VS5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:35.897112773 (282h21m34.448583234s) [noise=true , address=81.171.14.70:4031 , piece_id=LYWET76RCJLTUN3DJDSVMDOZ6X5S53KPZ264P7D5FS7XCRDSYIQA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.027285292 (282h21m34.578755743s) [noise=true , address=81.171.14.70:4045 , piece_id=EOJUCJVNYJIF447J6FKIRVIXM3IJ5OU6TEVUKHFKQCUUVIMBR2EA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.070911284 (282h21m34.622381745s) [noise=true , address=89.149.201.147:4011 , piece_id=F5544PH4I4Z224P2DVMAXLM57DWIFT37KKXUJBR7B3OG5FV7KK6A, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.073062550 (282h21m34.624533011s) [noise=true , address=89.149.201.147:4018 , piece_id=LPC2AQ7TX4UUEI4I5VMGDOIFJI7R3GJGAOL5Q2O7SVY42OFN3XVA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.083744591 (282h21m34.635215052s) [noise=true , address=89.149.201.147:4008 , piece_id=GSCXEJOMPWFECP7YSD66BRY26ZTXGIRZIMZYWKIW2SLRGPSJT7OQ, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.146992305 (282h21m34.698462766s) [noise=true , address=89.149.201.147:4003 , piece_id=SDOIACESWFHPNPOZUZIXW5F2K62OUKCLCBHUKD4YKPLUQG4XXE5Q, seg_pos={0 0}, upload=248651]: Done uploading piece. err: <nil> Nov 21 02:25:36.147018144 (282h21m34.698488605s) [seg_pos={0 0}, upload=248651]: Segment reached optimal threshold of 26 pieces. Nov 21 02:25:36.147392256 (282h21m34.698862717s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz): protocol: expected piece hash; context canceled; EOF Nov 21 02:25:36.147403267 (282h21m34.698873728s) [noise=true , address=89.149.201.147:4021 , piece_id=YPXYWJWR2LMLCA4PFNLVHI4HBZ3ZP6VGIHCMAHVOADM5USZ3UJWA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148140441 (282h21m34.699610933s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9): protocol: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53374->89.149.201.147:4010: use of closed network connection Nov 21 02:25:36.148153566 (282h21m34.699624027s) [noise=true , address=89.149.201.147:4010 , piece_id=X2Y6OVKDJOUA7DUWJAJAEN2FWYOFC3XXJPLNHCFQCXIV543RCIAA, seg_pos={0 0}, upload=248651]: Long tail cancellation due to optimal threshold Nov 21 02:25:36.148213310 (282h21m34.699683761s) [seg_pos={0 0}, upload=248651]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 21 02:25:36.148357414 (282h21m34.699827875s) [upload=248651]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 21 02:25:36.148395076 (282h21m34.699865547s) [upload=248651]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 21 02:25:36.406626090 (282h21m34.958096551s) [upload=248651]: Done uploading segments... Nov 21 02:25:36.406651539 (282h21m34.958122s) [upload=248651]: Upload finished. err: <nil> Duration: 6.143358447s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)