Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.....X...] duration: 7.772450337s trace: https://jaeger.dev.storj.io/trace/62a0bcb1c23573f5 FAIL (2025-11-21 14:12:19 UTC) orbiter warning: context deadline exceeded Duration: 7.772512146s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.836879418s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:12:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.836909494s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 21 out of 60 is failed (threshold=6) [...............X.X.X......X...X..X..XX.XXX..X..X.X..X.XXXXXX] duration: 5.001465805s trace: https://jaeger.dev.storj.io/trace/134250f0ed42f1a2 FAIL (2025-11-21 14:12:05 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001494911s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.....X..] duration: 8.726732236s trace: https://jaeger.dev.storj.io/trace/62a0bcb1c23573f5 FAIL (2025-11-21 14:11:50 UTC) orbiter warning: context deadline exceeded Duration: 8.726773687s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.879197839s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:11:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.879227194s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.....X.] duration: 15.903179585s trace: https://jaeger.dev.storj.io/trace/62a0bcb1c23573f5 FAIL (2025-11-21 14:11:17 UTC) orbiter warning: context deadline exceeded Duration: 15.903314593s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 21 out of 60 is failed (threshold=6) [X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.XXXXX] duration: 5.002008511s trace: https://jaeger.dev.storj.io/trace/6a193315394577ec FAIL (2025-11-21 14:10:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002040053s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.929927929s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:10:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.929944171s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.933633166s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:10:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.9336621s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration: 30.000921616s trace: https://jaeger.dev.storj.io/trace/62a0bcb1c23573f5 FAIL (2025-11-21 14:10:36 UTC) orbiter warning: context deadline exceeded Duration: 30.000945011s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 20 out of 60 is failed (threshold=6) [.X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.XXXX] duration: 5.001568322s trace: https://jaeger.dev.storj.io/trace/473cd1b0295e67df FAIL (2025-11-21 14:10:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001607166s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 19 out of 60 is failed (threshold=6) [..X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.XXX] duration: 5.001782361s trace: https://jaeger.dev.storj.io/trace/2258b8797a78833a FAIL (2025-11-21 14:10:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001823679s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.238654751s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:09:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.238687703s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 10.957233721s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:09:24 UTC) orbiter warning: context deadline exceeded Duration: 10.957264298s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [...X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.XX] duration: 5.001988522s trace: https://jaeger.dev.storj.io/trace/404c8145582b752e FAIL (2025-11-21 14:09:16 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002015465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.285104833s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:09:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.285139808s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [....X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.X] duration: 5.001517186s trace: https://jaeger.dev.storj.io/trace/2f9245cb6e5d9687 FAIL (2025-11-21 14:08:46 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001546101s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 16.420653973s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:08:31 UTC) orbiter warning: context deadline exceeded Duration: 16.420707966s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.296534901s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:08:06 UTC) orbiter warning: context deadline exceeded Duration: 7.296563507s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.309449261s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:08:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.309484568s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.814083921s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:07:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.814112887s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [.....X...............X.X.X......X...X..X..XX.XXX..X..X.X..X.] duration: 3.667239826s trace: https://jaeger.dev.storj.io/trace/5fe37361e8e13088 FAIL (2025-11-21 14:07:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.667270284s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [X.....X...............X.X.X......X...X..X..XX.XXX..X..X.X..X] duration: 5.000977347s trace: https://jaeger.dev.storj.io/trace/5fe37361e8e13088 FAIL (2025-11-21 14:07:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001110821s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 12.57455083s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:07:07 UTC) orbiter warning: context deadline exceeded Duration: 12.574595916s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.77909278s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:06:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.77913526s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.210101792s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:06:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.21014805s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 10.948752721s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:06:09 UTC) orbiter warning: context deadline exceeded Duration: 10.948827002s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [.X.....X...............X.X.X......X...X..X..XX.XXX..X..X.X..] duration: 4.924223623s trace: https://jaeger.dev.storj.io/trace/31b2885db10719ca FAIL (2025-11-21 14:06:06 UTC) orbiter warning: context deadline exceeded Duration: 4.924363449s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001155311s trace: https://jaeger.dev.storj.io/trace/6f8bc6a4eac5272e FAIL (2025-11-21 14:05:43 UTC) orbiter warning: context deadline exceeded Duration: 30.00118028s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [X.X.....X...............X.X.X......X...X..X..XX.XXX..X..X.X.] duration: 3.692061382s trace: https://jaeger.dev.storj.io/trace/31b2885db10719ca FAIL (2025-11-21 14:05:26 UTC) orbiter warning: context deadline exceeded Duration: 3.692093611s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.382117981s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:05:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.382146866s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [XX.X.....X...............X.X.X......X...X..X..XX.XXX..X..X.X] duration: 5.001835732s trace: https://jaeger.dev.storj.io/trace/31b2885db10719ca FAIL (2025-11-21 14:04:31 UTC) orbiter warning: context deadline exceeded Duration: 5.001858756s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.141929368s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:04:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.141959806s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [.XX.X.....X...............X.X.X......X...X..X..XX.XXX..X..X.] duration: 5.001039646s trace: https://jaeger.dev.storj.io/trace/1763a18620ed7939 FAIL (2025-11-21 14:03:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001069193s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.745443397s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:03:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.745472492s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.341118527s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:03:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.341222075s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [..XX.X.....X...............X.X.X......X...X..X..XX.XXX..X..X] duration: 5.00117493s trace: https://jaeger.dev.storj.io/trace/1763a18620ed7939 FAIL (2025-11-21 14:02:56 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001199678s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 7.950112689s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:02:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 7.950144689s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [X..XX.X.....X...............X.X.X......X...X..X..XX.XXX..X..] duration: 4.122827329s trace: https://jaeger.dev.storj.io/trace/4c40bbaccb45f651 FAIL (2025-11-21 14:02:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.122855725s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.084298566s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:01:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.084326619s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [.X..XX.X.....X...............X.X.X......X...X..X..XX.XXX..X.] duration: 3.814205998s trace: https://jaeger.dev.storj.io/trace/4c40bbaccb45f651 FAIL (2025-11-21 14:00:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.81422339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.380579325s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 14:00:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.380612048s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [X.X..XX.X.....X...............X.X.X......X...X..X..XX.XXX..X] duration: 5.002885128s trace: https://jaeger.dev.storj.io/trace/4c40bbaccb45f651 FAIL (2025-11-21 14:00:10 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.002979648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 7.965138472s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 13:59:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 7.965165213s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [.X.X..XX.X.....X...............X.X.X......X...X..X..XX.XXX..] duration: 5.001424159s trace: https://jaeger.dev.storj.io/trace/7ae4a4e06f2ab150 FAIL (2025-11-21 13:59:10 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001442462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.863178326s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 13:59:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.863210277s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [..X.X..XX.X.....X...............X.X.X......X...X..X..XX.XXX.] duration: 3.792150558s trace: https://jaeger.dev.storj.io/trace/7ae4a4e06f2ab150 FAIL (2025-11-21 13:58:04 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.792187568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.150042808s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 13:57:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 6.150060993s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [X..X.X..XX.X.....X...............X.X.X......X...X..X..XX.XXX] duration: 5.001474643s trace: https://jaeger.dev.storj.io/trace/7ae4a4e06f2ab150 FAIL (2025-11-21 13:57:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001499911s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.853423972s trace: https://jaeger.dev.storj.io/trace/2c4b010d23e46d91 FAIL (2025-11-21 13:56:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 24) ======LOGS====== AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291418442 (293h31m7.842888914s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.291954664 (293h31m7.843425145s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.293302360 (293h31m7.844772851s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.294979363 (293h31m7.846449854s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.295760250 (293h31m7.847230711s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.300934223 (293h31m7.852404674s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.301942341 (293h31m7.853412814s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.302417767 (293h31m7.853888218s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:09.304119598 (293h31m7.855590059s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Uploading piece... Nov 21 13:35:10.220344333 (293h31m8.771814794s) [upload=258480]: Next returned nil segment Nov 21 13:35:10.220363840 (293h31m8.771834301s) [upload=258480]: Waiting for error group managing segments... Nov 21 13:35:10.536640979 (293h31m9.08811144s) [noise=true , address=185.167.204.205:4027 , piece_id=7C2DDLUHJQCI62D2YTI76NDABUZRSPK3FIZ7N45OG5GX3SZWCHNA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.537115091 (293h31m9.088585552s) [noise=true , address=185.167.204.205:4020 , piece_id=LPUWKZY6RPB7TGO22BL4YRA7ETB2RHWUFOXPJX7TZE3OHSFXEH7A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539139225 (293h31m9.090609686s) [noise=true , address=185.167.204.206:4056 , piece_id=7VD7D3HW5YIZUYRZ5LWEB4VDXN2R6OEU7CTCGLKZUVKEC44V743Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.539668604 (293h31m9.091139065s) [noise=true , address=185.167.204.206:4048 , piece_id=6DBNGIBFWN5PPV72YWVRSSQVYF2AZ7EHS55PJDLDLGZEGXRYYHBA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.541238333 (293h31m9.092708794s) [noise=true , address=185.167.204.205:4008 , piece_id=UE3HOAD4KZRUWKY63RDOZ5LQ7VYAQPNV653CNSWCRXD2WL5MR27A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.542375438 (293h31m9.093845919s) [noise=true , address=185.167.204.205:4036 , piece_id=F652JXWWTK5XUJBY5SSU6IKKJE72DGITQXBRYK3LMQEF63DMRIEQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543153891 (293h31m9.094624342s) [noise=true , address=185.167.204.205:4002 , piece_id=YFXJFZTNYWU3AZXQYDU5VBWJCPRVBCYCANXPQ77UGR5ZBY5VV53Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543618846 (293h31m9.095089307s) [noise=true , address=185.167.204.206:4047 , piece_id=OHARKEAXGZLVG5XHD6RYKKWS57JFGH4JR2TAFIQSPUXTRMJD4WZA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.543662328 (293h31m9.095132789s) [noise=true , address=185.167.204.206:4066 , piece_id=T5X6MZEPB6P3DZ2HVMXCC4XJX5JJFXKGASJRWJDCK5TBHDEA2OLQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544259055 (293h31m9.095729516s) [noise=true , address=185.167.204.205:4011 , piece_id=O2KXUNUHCKAQZSL5BJRRRC6IIZTSXHM74OZ7REQI3RGSMFA3PN6Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.544805565 (293h31m9.096276016s) [noise=true , address=185.167.204.206:4060 , piece_id=VJPM3YAQI5N2VTEBJ5GZ75JCCBOQS5IBEFUHEWTOX3BAKSP7TWFQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545043088 (293h31m9.096513559s) [noise=true , address=185.167.204.205:4007 , piece_id=R6HJTNG5VAAMG2Y6YXK4K4YBQFUSAJQ4774UCOSGJSRDM4RWLICA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:10.545385921 (293h31m9.096856382s) [noise=true , address=185.167.204.205:4034 , piece_id=H66BDBLG4T2BABKBNKMJLY6QPZT2VVSZC3GW6F5V2L2BBFOWE76Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.759454013 (293h31m10.310924484s) [noise=true , address=89.149.201.147:4006 , piece_id=T2KROQWQR5LXKBJEJO6LICLBP5CUBETT3TOACRQCGYXKZ364TYMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.779325021 (293h31m10.330795482s) [noise=true , address=89.149.201.147:4015 , piece_id=VIFNKNEVMCI2CFBREZOLVGFJ2X7Y7ZX6FWDC6PO6CLVX2LFW4FYQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.797229894 (293h31m10.348700345s) [noise=true , address=89.149.201.147:4021 , piece_id=K3ZPQXTSSUGIGIX33FCYOC6FRCTWDBRVRHUHHGEJLNWFCJFCIT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.804823945 (293h31m10.356294406s) [noise=true , address=89.149.201.147:4012 , piece_id=3QAKWEG37QPAE4AORUSYURK47YIOTM5VHC563QWC43UGGPS7UV7Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.828388066 (293h31m10.379858537s) [noise=true , address=89.149.201.147:4014 , piece_id=MM2F7NMVCD3AFX2UM7FRGSO5ZD2OTJPXJEUVKJ63IL5L74LEXZSQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:35:11.848656089 (293h31m10.40012655s) [noise=true , address=89.149.201.147:4001 , piece_id=OGRBX7OEEUPFXWU3EOYD44FGBQ7FQ5CHHZ2RVRHFJGOQ74EK7K5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.180750503 (293h32m0.732220964s) [noise=true , address=81.171.14.70:4028 , piece_id=ESUHXDH256EM67DXYQ7O5THLHNAPPJ6I3ARZC5D3GSLJD7CNJV5A, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:02.321725695 (293h32m0.873196156s) [noise=true , address=81.171.14.70:4044 , piece_id=KMR7RCY3AFLFIAJWIQXYC6ZSX7QLL6OKSCQMYHDKKCRX4MZAC4TQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:03.575396105 (293h32m2.126866556s) [noise=true , address=81.171.14.70:4041 , piece_id=N36O2DAKHTDWU4BPERULQHIUWQQDM5TQUQD7Y36XDA7SPPQML2LQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:05.676531843 (293h32m4.228002294s) [noise=true , address=81.171.14.70:4047 , piece_id=Q4N3CMZGTJGYFS35Y75GJHLQLEYMRIX4SSGTJXEHCCXIAX5Y2T3Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.329832254 (293h32m6.881302715s) [noise=true , address=81.171.14.70:4043 , piece_id=TQCRSBAQ4AJPCVP2JZYD4HQF375R2K7P5AIMCZW5EJVFJYUGKYGA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: <nil> Nov 21 13:36:08.628985817 (293h32m7.180456278s) [noise=true , address=81.171.14.70:4042 , piece_id=E5AMRUQ7TS2R6K7ILJ2YSG7UNGO4P5PFTOMHVF6FXOOIWW46Y2NA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 188.166.225.170:40010->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 13:36:08.629102389 (293h32m7.18057285s) [noise=true , address=81.171.14.70:4048 , piece_id=NUU26IUS2OGXAEVLUDEYTXJL4E6GZM3M7MP5BNBI6KTRWQ2RZT4Q, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55932->81.171.14.70:4048: use of closed network connection Nov 21 13:36:08.629502702 (293h32m7.180973163s) [noise=true , address=81.171.14.70:4031 , piece_id=LJW25GLWHJCV5CFUITJ5CPYDNWOFG3NBK4A5NUX2VVV4CYCDHWMQ, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58232->81.171.14.70:4031: use of closed network connection Nov 21 13:36:08.629974720 (293h32m7.181445181s) [noise=true , address=81.171.14.70:4026 , piece_id=ATEKV2WY2WDNBLY7W4THWSO52AJ76Q3XVT7QDGT3EBMPWVRL5EKA, seg_pos={0 0}, upload=258480]: Done uploading piece. err: ecclient: upload failed (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b, address:81.171.14.70:4026): protocol: write tcp 188.166.225.170:41806->81.171.14.70:4026: use of closed network connection; context deadline exceeded Nov 21 13:36:08.630048300 (293h32m7.181518751s) [seg_pos={0 0}, upload=258480]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil> Nov 21 13:36:08.800451489 (293h32m7.35192194s) [upload=258480]: Done uploading segments... Nov 21 13:36:08.800522093 (293h32m7.351992554s) [upload=258480]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24) Duration: 5.853450682s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)