Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.463593946s FAIL (2025-09-06 09:44:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== , seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.936915408 (187h36m5.617110997s) [noise=true , address=77.90.31.10:4518 , piece_id=STLFIUHUOXVMVIUI64FQH27KRGALTYULOT56RRHGEKU7VH6MLCUQ, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.937396308 (187h36m5.617591877s) [noise=true , address=45.91.116.5:4299 , piece_id=MLHK463D35BUAE6YJA54NFP4PMVAA5EZPNGR6Q2GDEIV37PPLO7Q, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.937719747 (187h36m5.617915316s) [noise=true , address=45.91.116.9:4262 , piece_id=ONC47H4S6GXLHBZTYBQNPNIHR23S47PSJOR6I2ZXY7IRYXWTNC3A, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.938080377 (187h36m5.618275936s) [noise=true , address=45.91.116.11:4118 , piece_id=IRHAOG733XCECKDA6KUDK45OUWRJESC2YC46LGWCGLJAYEHIS43Q, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.938332630 (187h36m5.618528199s) [noise=true , address=77.90.31.19:4842 , piece_id=TPKAVHUII7ZHNSF4F2ERPYZQM445EXXYPRANE4RM2O6XH2GSWO4A, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.938730932 (187h36m5.618926501s) [noise=true , address=45.91.116.4:4937 , piece_id=XUXJQZ7C4Y5DVZMXH43LUOP5YFORRZJV5R3UH4TTCW556T5QFSMQ, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.939062596 (187h36m5.619258165s) [noise=true , address=45.91.116.12:4226 , piece_id=35NYHH6FIJP6F3RMQJM6SZZBSOYMPPAUEWMMKRWPJGXYTEZ4X7YQ, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.939277027 (187h36m5.619472596s) [noise=true , address=45.91.116.9:4263 , piece_id=GV44ZHPYFOH7ZR2TIXPEDQTCCALZMR5KUKOIRZND2XFV2QD7ARNA, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.939570509 (187h36m5.619766078s) [noise=true , address=45.91.116.10:4082 , piece_id=S4WY5T32WW4WJHDZCCTE75GPPLCKXXHR6TVMZW72KKOCEHKFF2EQ, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:08.939799527 (187h36m5.619995096s) [noise=true , address=45.91.116.12:4227 , piece_id=S2V34DSCBETAKT5H7GDPD2HPHPCWYDPBLH2I2IMAIXN5XYBF2SPA, seg_pos={0 0}, upload=165074]: Uploading piece... Sep 6 09:44:23.017057264 (187h36m19.697252833s) [upload=165074]: Next segment err: context deadline exceeded Sep 6 09:44:23.018188088 (187h36m19.698383657s) [noise=true , address=45.91.116.9:4262 , piece_id=ONC47H4S6GXLHBZTYBQNPNIHR23S47PSJOR6I2ZXY7IRYXWTNC3A, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 188.166.225.170:38854->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018226021 (187h36m19.69842159s) [noise=true , address=45.91.116.4:4937 , piece_id=XUXJQZ7C4Y5DVZMXH43LUOP5YFORRZJV5R3UH4TTCW556T5QFSMQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: read tcp 188.166.225.170:34086->45.91.116.4:4937: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018310001 (187h36m19.69850557s) [noise=true , address=45.91.116.13:4155 , piece_id=26C7VNB33FOVFXIBBSUZNCQE27XDUC3GQI3ONQDVPPQCRGPYHXLQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 188.166.225.170:55954->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018361129 (187h36m19.698556698s) [noise=true , address=77.90.31.19:4843 , piece_id=Z4PMGWA42VP7NRWC6QDSTOFGNYCV3DHYVTJLC2OUFV7HRQPNCR4A, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 188.166.225.170:37010->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018427326 (187h36m19.698622895s) [noise=true , address=45.91.116.12:4226 , piece_id=35NYHH6FIJP6F3RMQJM6SZZBSOYMPPAUEWMMKRWPJGXYTEZ4X7YQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 188.166.225.170:47260->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018476420 (187h36m19.698671989s) [noise=true , address=45.91.116.10:4082 , piece_id=S4WY5T32WW4WJHDZCCTE75GPPLCKXXHR6TVMZW72KKOCEHKFF2EQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:37444->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018509343 (187h36m19.698704912s) [noise=true , address=45.91.116.8:4045 , piece_id=SS44KR2QGVT6HR4UWFMYM67DGUOLGSUA2HDH5O6B2LSSXXVP73UQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 188.166.225.170:33456->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018540502 (187h36m19.698736071s) [noise=true , address=45.91.116.5:4299 , piece_id=MLHK463D35BUAE6YJA54NFP4PMVAA5EZPNGR6Q2GDEIV37PPLO7Q, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 188.166.225.170:57192->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018552696 (187h36m19.698748265s) [noise=true , address=45.91.116.9:4263 , piece_id=GV44ZHPYFOH7ZR2TIXPEDQTCCALZMR5KUKOIRZND2XFV2QD7ARNA, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: rpc: dial tcp 45.91.116.9:4263: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018606198 (187h36m19.698801767s) [noise=true , address=77.90.31.10:4518 , piece_id=STLFIUHUOXVMVIUI64FQH27KRGALTYULOT56RRHGEKU7VH6MLCUQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 188.166.225.170:41046->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018654179 (187h36m19.698849738s) [noise=true , address=77.90.31.16:4735 , piece_id=UKWBQHGRKZESFJJB3TPQAG57CNDR6AJQNGEUH4H365QPMSJPIZIQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018666353 (187h36m19.698861922s) [noise=true , address=45.91.116.8:4048 , piece_id=3SC4Q7TMZUS5BLBHIMKGE5PZFEZPF5L5I4HPWASAAZGJDJO3WF2Q, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:48268->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018700288 (187h36m19.698895857s) [noise=true , address=77.90.31.19:4842 , piece_id=TPKAVHUII7ZHNSF4F2ERPYZQM445EXXYPRANE4RM2O6XH2GSWO4A, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: read tcp 188.166.225.170:39862->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018704516 (187h36m19.698900085s) [noise=true , address=45.91.116.7:4010 , piece_id=F6ROLCJAJT3XUM6JP26MNREDSOZN35M5TT5D2AWZFBNUNQWVQFTQ, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: read tcp 188.166.225.170:51600->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018734012 (187h36m19.698929581s) [noise=true , address=45.91.116.11:4118 , piece_id=IRHAOG733XCECKDA6KUDK45OUWRJESC2YC46LGWCGLJAYEHIS43Q, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 188.166.225.170:33370->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018737981 (187h36m19.69893355s) [noise=true , address=45.91.116.12:4227 , piece_id=S2V34DSCBETAKT5H7GDPD2HPHPCWYDPBLH2I2IMAIXN5XYBF2SPA, seg_pos={0 0}, upload=165074]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 188.166.225.170:48848->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:23.018752728 (187h36m19.698948297s) [seg_pos={0 0}, upload=165074]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 30.463610801s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [..............................................X.X..X.XX.....] duration: 20.474233779s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:44:22 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 20.474250219s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.................................X.X.X.X.XX.XXX.X..X...X..X.] duration: 20.534578881s FAIL (2025-09-06 09:44:15 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 20.534606293s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [.................................XXXXXX.....XXX.X.X.XXXX....] duration: 30.000988306s FAIL (2025-09-06 09:44:12 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:40:55.687477176 (187h32m52.367672745s) [upload=165027]: Starting upload Sep 6 09:40:55.687529276 (187h32m52.367724835s) [upload=165027]: Done starting upload Sep 6 09:40:55.687623947 (187h32m52.367819516s) [upload=165027]: Uploading segments... Sep 6 09:40:55.687661369 (187h32m52.367856938s) [upload=165027]: Got next segment. Inline: true Sep 6 09:40:55.687665246 (187h32m52.367860815s) [upload=165027]: Waiting for error group managing segments... Sep 6 09:40:55.687720752 (187h32m52.367916321s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:40:55.687725201 (187h32m52.36792077s) [upload=165027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:40:55.687728007 (187h32m52.367923576s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:41:17.325212667 (187h33m14.005408236s) [upload=165027]: Done uploading segments... Sep 6 09:41:17.325231412 (187h33m14.005426981s) [upload=165027]: Upload finished. err: <nil> Duration:* 30.001016581s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 3.674627601s FAIL (2025-09-06 09:44:10 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.67465335s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 30.001090872s FAIL (2025-09-06 09:44:09 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 30.001104257s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 1m0.000602903s FAIL (2025-09-06 09:44:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000647999s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 1m1.518787729s FAIL (2025-09-06 09:44:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== 48862 (187h36m0.000644431s) [noise=true , address=77.90.31.17:4797 , piece_id=TTE7JMPTS6AO3YVGR4IW4DVHDAROFW3PG7Q27Q6DUNF7OB6TJ2QA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: write tcp 188.166.225.170:35412->77.90.31.17:4797: use of closed network connection; write tcp 188.166.225.170:35412->77.90.31.17:4797: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35412->77.90.31.17:4797: use of closed network connection Sep 6 09:44:03.320816244 (187h36m0.001011813s) [noise=true , address=77.90.31.14:4665 , piece_id=WCUTCWQANESKUNSD6WEORNYXO55DFU33LBHMMPKNW7DT276F6UXQ, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce, address:77.90.31.14:4665): protocol: write tcp 188.166.225.170:41948->77.90.31.14:4665: use of closed network connection; write tcp 188.166.225.170:41948->77.90.31.14:4665: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41948->77.90.31.14:4665: use of closed network connection Sep 6 09:44:03.320871410 (187h36m0.00106698s) [noise=true , address=145.79.227.5:4255 , piece_id=A2ULNVRTQ3U7WOSMYUGBNMENMLCZPSBHXKRWXJV2AETQ466PEBGA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12fTNKdc6AZbkfsKRFZwTwV6wPz9j9ZD9t2eNkHBJNLMNcGh9gR, address:145.79.227.5:4255): protocol: write tcp 188.166.225.170:35038->145.79.227.5:4255: use of closed network connection; write tcp 188.166.225.170:35038->145.79.227.5:4255: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35038->145.79.227.5:4255: use of closed network connection Sep 6 09:44:03.320908321 (187h36m0.00110389s) [noise=true , address=145.79.227.4:4188 , piece_id=QWPKKS34VVEWSBN3QS335FB6Q2CT5KAE4AR5MC3QZJ3JBLHTWH3A, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: write tcp 188.166.225.170:50578->145.79.227.4:4188: use of closed network connection; write tcp 188.166.225.170:50578->145.79.227.4:4188: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50578->145.79.227.4:4188: use of closed network connection Sep 6 09:44:03.320942747 (187h36m0.001138316s) [noise=true , address=145.79.227.40:4061 , piece_id=SIOJVPPOGHK2KYA5G7LQOASBQSCBNKH7NZO3CZRSMKLDP2WLSQ5Q, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: write tcp 188.166.225.170:50588->145.79.227.40:4061: use of closed network connection; write tcp 188.166.225.170:50588->145.79.227.40:4061: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50588->145.79.227.40:4061: use of closed network connection Sep 6 09:44:03.320979627 (187h36m0.001175196s) [noise=true , address=77.90.31.18:4829 , piece_id=CSL46N4EMWLRL3MXNWO437J3MUBWLPQ6L3545YDHOYZBZAJU3GQQ, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: write tcp 188.166.225.170:35896->77.90.31.18:4829: use of closed network connection; write tcp 188.166.225.170:35896->77.90.31.18:4829: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35896->77.90.31.18:4829: use of closed network connection Sep 6 09:44:03.321078967 (187h36m0.001274526s) [noise=true , address=145.79.227.18:4022 , piece_id=DOLRDOS62HL2SPYZEFD2UTWU4T4I2RNKLNUEQL5EYSO7ES3FUN2A, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1XGBWwhqJ324UTpv3b1bNsGVUjJSLptvqV6dHrYJsto8KbDvYU, address:145.79.227.18:4022): protocol: read tcp 188.166.225.170:58676->145.79.227.18:4022: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:03.321110147 (187h36m0.001305716s) [noise=true , address=45.91.116.8:4049 , piece_id=WDWWZ6SUJMIR4NJQWEPUKJGAOOTZ5VYARYGQ633AGVGRPDQ26FFA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12bbyRMajztw9frbd1V1meid4vQU1AC343NtZrg6qe27HUyw6bQ, address:45.91.116.8:4049): protocol: write tcp 188.166.225.170:33426->45.91.116.8:4049: use of closed network connection; write tcp 188.166.225.170:33426->45.91.116.8:4049: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33426->45.91.116.8:4049: use of closed network connection Sep 6 09:44:03.321289681 (187h36m0.00148525s) [noise=true , address=145.79.227.41:4117 , piece_id=DSPQSSPDJOHJ5BBPE2ASYOE3AU33LDHCN2X2QU24KPTI7ILDRHIA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:145.79.227.41:4117): protocol: read tcp 188.166.225.170:50358->145.79.227.41:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:03.321307294 (187h36m0.001502883s) [noise=true , address=45.91.116.6:4004 , piece_id=6VRBYUCWIM6GMTZTRJBCGGQMQHMKZS4VJSJ4M2GS2TMAA7O6FLVA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1gC8PAH3JvFmoFrvSAF5xiBhQDMZ1ScfxuxVBrNQb5A1odWsvX, address:45.91.116.6:4004): protocol: write tcp 188.166.225.170:32862->45.91.116.6:4004: use of closed network connection; write tcp 188.166.225.170:32862->45.91.116.6:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32862->45.91.116.6:4004: use of closed network connection Sep 6 09:44:03.321337403 (187h36m0.001532972s) [noise=true , address=145.79.227.52:4472 , piece_id=LXGYQF27ZFX7CFSVEET6W4EGTC7HINOOY7NOZNOI67VQKPKNQ7DA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12otbB1RmPUKSKo813hrKPQVAyeqmhjhApxi6ndtHBCfn9bGGct, address:145.79.227.52:4472): protocol: write tcp 188.166.225.170:43160->145.79.227.52:4472: use of closed network connection; write tcp 188.166.225.170:43160->145.79.227.52:4472: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43160->145.79.227.52:4472: use of closed network connection Sep 6 09:44:03.321388439 (187h36m0.001584008s) [noise=true , address=145.79.227.14:4849 , piece_id=4SYHOB4CVA3L7FAAXBRRDDT7EOY6C5K7P65WE3KAGRJZFQRXLRKA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1n2gSzVmNCEZcfVsFKibjXHs2jMr82D3SWQEykFQQ4uyhhnrj1, address:145.79.227.14:4849): protocol: write tcp 188.166.225.170:36680->145.79.227.14:4849: use of closed network connection; write tcp 188.166.225.170:36680->145.79.227.14:4849: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36680->145.79.227.14:4849: use of closed network connection Sep 6 09:44:03.321424439 (187h36m0.001620008s) [noise=true , address=145.79.227.48:4326 , piece_id=EPC2BTVZWQEG3NEAZFWDRJ7GVKTRTW4IGKHXRBAEO7X36JTNK5JQ, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12Ys2boqq8tJDTRRSGV47s1uhZqA68LzZuQ3HwpKwghUPc5Dwy7, address:145.79.227.48:4326): protocol: read tcp 188.166.225.170:37054->145.79.227.48:4326: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:03.321537475 (187h36m0.001733044s) [noise=true , address=145.79.227.33:4903 , piece_id=2AVBGIG2OLQU4LGFPWBITFBR34E6JYEL2VHQYPDIRWIN5QFIOIDQ, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12mY4YcLLVbnVuXtKSHwyv4sgTycqzxUZHGVQha5WfXXTPjNGLq, address:145.79.227.33:4903): protocol: write tcp 188.166.225.170:47452->145.79.227.33:4903: use of closed network connection; write tcp 188.166.225.170:47452->145.79.227.33:4903: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47452->145.79.227.33:4903: use of closed network connection Sep 6 09:44:03.321568895 (187h36m0.001764464s) [noise=true , address=77.90.31.11:4588 , piece_id=VDHWSHQRJNQZR5ZEZVR2J5IFPIGOFUJEAQRL2NDWEAIACUECXNLA, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12H6EaBk5vQxDeccjDtyoJmuANkspKuRPCyS7qHuDn16GQPa8o6, address:77.90.31.11:4588): protocol: read tcp 188.166.225.170:37394->77.90.31.11:4588: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:03.321577902 (187h36m0.001773481s) [noise=true , address=145.79.227.13:4813 , piece_id=L6OIU5I2MHKBGPQEGCL5OCKICETE4JRLHD6FQKI7CCU5IB6NMCJQ, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:12EHHTcQmhaHswSx1ubyHmPTHBB5ptuxmydbKHLjYChF1mudMSB, address:145.79.227.13:4813): protocol: write tcp 188.166.225.170:46020->145.79.227.13:4813: use of closed network connection; write tcp 188.166.225.170:46020->145.79.227.13:4813: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46020->145.79.227.13:4813: use of closed network connection Sep 6 09:44:03.321603180 (187h36m0.001798749s) [noise=true , address=145.79.227.37:4571 , piece_id=XJBVP4E3TQ2SPA5DIJFRANHJUZB73U7RTWVOC5NB3THX3XF34X3Q, seg_pos={0 0}, upload=165057]: Done uploading piece. err: ecclient: upload failed (node:1kL49w1XE8S2Dgk9RUfjuXFxisMW3BUEPxPZR87TCVBpAwbyit, address:145.79.227.37:4571): protocol: read tcp 188.166.225.170:60230->145.79.227.37:4571: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:44:03.321631064 (187h36m0.001826633s) [seg_pos={0 0}, upload=165057]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m1.518812055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 7 out of 60 is failed (threshold=3) [................................X..X....X.X........X.X....X.] duration: 5.536327537s FAIL (2025-09-06 09:44:04 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48) Duration: 5.536356s core uplink eu1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [...............................................X.X..X.XX....] duration: 43.405336015s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:44:01 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 43.405376512s edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.................................X..X....X.X........X.X....X] duration: 30.171943985s FAIL (2025-09-06 09:43:54 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 48) Duration: 30.171968872s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 31.509801646s FAIL (2025-09-06 09:43:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== oad=165063]: Uploading piece... Sep 6 09:43:44.919781669 (187h35m41.599977239s) [noise=true , address=45.91.116.8:4048 , piece_id=BG3N7OLYW2MSDXMQ5RD44O62F6FX3WRIIFDHYIWHF6TU6SR5JGKQ, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.919920134 (187h35m41.600115703s) [noise=true , address=77.90.31.21:4877 , piece_id=M7UYC2GF5AWJSJTJYKAL3T5YXIKG76636FRVTYTOPEZFWIWRCH6Q, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920055233 (187h35m41.600250802s) [noise=true , address=45.91.116.11:4118 , piece_id=OGS7IXOMEBXJ32W5C5BYBE5JU6XHRUDG3Q5V2KL4VIQLRM6KPYLA, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920169521 (187h35m41.600365091s) [noise=true , address=45.91.116.10:4081 , piece_id=KQSEGVPI3YFRBMYO4HMIC4HY7LEYLBGNWFWDAI3BUJTUXOXFSP3Q, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920363262 (187h35m41.600558831s) [noise=true , address=77.90.31.17:4771 , piece_id=4E5BZLNWYINZ5TU2UFGLQGC6RGUIWKQ4M2O2PJ3WDKJTH5KVQPXQ, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920445980 (187h35m41.600641579s) [noise=true , address=45.91.116.11:4117 , piece_id=J346ONZBJHSXOXAJ2VHDFJG64UD5RJLPEQR54WRKWQHLIHJK7RLA, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920657515 (187h35m41.600853084s) [noise=true , address=77.90.31.14:4663 , piece_id=IOSCP6OQZYCOPIHBENY4CKSR5AHMDGKMWTBY4O2PBHDMHROX5ANQ, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.920836287 (187h35m41.601031856s) [noise=true , address=45.91.116.4:4940 , piece_id=M74OQEDVUP5B2URUD6PJRC2HXL3SCP3Q75ZH66GKW65BGXDEWHCQ, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:44.921091015 (187h35m41.601286584s) [noise=true , address=77.90.31.16:4735 , piece_id=47DZHGQGJNVFKPVMFFS5UWFUOIBICETL55CLSNBQSXM7TS2DSPXA, seg_pos={0 0}, upload=165063]: Uploading piece... Sep 6 09:43:51.264025557 (187h35m47.944221126s) [upload=165063]: Next segment err: context deadline exceeded Sep 6 09:43:51.264635423 (187h35m47.944830992s) [noise=true , address=45.91.116.14:4191 , piece_id=AGHHDSLPUAL5K3AKVJWC3KEBJKS6VKKMY4TIAW5L2M3H65NGKGZA, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 188.166.225.170:57318->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.264751516 (187h35m47.944947085s) [noise=true , address=45.91.116.4:4937 , piece_id=RKRDD5N3X5CJQSQ7EGJY7PMZWG2EHZ3ON2LS4MQRNQO752TTVRDA, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: write tcp 188.166.225.170:34482->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:34482->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34482->45.91.116.4:4937: use of closed network connection Sep 6 09:43:51.264935818 (187h35m47.945131387s) [noise=true , address=45.91.116.8:4048 , piece_id=BG3N7OLYW2MSDXMQ5RD44O62F6FX3WRIIFDHYIWHF6TU6SR5JGKQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:44398->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265081007 (187h35m47.945276576s) [noise=true , address=45.91.116.4:4940 , piece_id=M74OQEDVUP5B2URUD6PJRC2HXL3SCP3Q75ZH66GKW65BGXDEWHCQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 188.166.225.170:60534->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265127254 (187h35m47.945322824s) [noise=true , address=45.91.116.7:4009 , piece_id=TVNM5ARMATAJ7VMQQZ7SDCQOBJLL3YZEV354L3UDVWEUVCETVYOQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 188.166.225.170:43884->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265164266 (187h35m47.945359835s) [noise=true , address=77.90.31.14:4663 , piece_id=IOSCP6OQZYCOPIHBENY4CKSR5AHMDGKMWTBY4O2PBHDMHROX5ANQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 188.166.225.170:36504->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265181879 (187h35m47.945377448s) [noise=true , address=45.91.116.9:4261 , piece_id=OFTRAIG3STUT5LKKSWEUGESIP6YHQQXOTNI34MNV75E5USLXFOZQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: read tcp 188.166.225.170:50288->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265210375 (187h35m47.945405934s) [noise=true , address=45.91.116.10:4082 , piece_id=45HPR7JACYEKLLX5GCKPG2EGD2QV3H7RJXX4DGTPBCHITAYYTH3Q, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: read tcp 188.166.225.170:51288->45.91.116.10:4082: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265225032 (187h35m47.945420601s) [noise=true , address=77.90.31.21:4877 , piece_id=M7UYC2GF5AWJSJTJYKAL3T5YXIKG76636FRVTYTOPEZFWIWRCH6Q, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: read tcp 188.166.225.170:48212->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265249248 (187h35m47.945444818s) [noise=true , address=77.90.31.14:4661 , piece_id=ON5UPC67KA2LJGKYP2XT3DAT2HEBT7STFMO7OQL7JXIXIQKLEXKQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 188.166.225.170:49064->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265270840 (187h35m47.945466409s) [noise=true , address=45.91.116.10:4081 , piece_id=KQSEGVPI3YFRBMYO4HMIC4HY7LEYLBGNWFWDAI3BUJTUXOXFSP3Q, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: read tcp 188.166.225.170:34730->45.91.116.10:4081: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265302741 (187h35m47.9454983s) [noise=true , address=45.91.116.11:4118 , piece_id=OGS7IXOMEBXJ32W5C5BYBE5JU6XHRUDG3Q5V2KL4VIQLRM6KPYLA, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: read tcp 188.166.225.170:60342->45.91.116.11:4118: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265316557 (187h35m47.945512126s) [noise=true , address=77.90.31.17:4771 , piece_id=4E5BZLNWYINZ5TU2UFGLQGC6RGUIWKQ4M2O2PJ3WDKJTH5KVQPXQ, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: read tcp 188.166.225.170:54672->77.90.31.17:4771: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265347928 (187h35m47.945543497s) [noise=true , address=45.91.116.11:4117 , piece_id=J346ONZBJHSXOXAJ2VHDFJG64UD5RJLPEQR54WRKWQHLIHJK7RLA, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:58540->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265354410 (187h35m47.945549979s) [noise=true , address=77.90.31.16:4735 , piece_id=47DZHGQGJNVFKPVMFFS5UWFUOIBICETL55CLSNBQSXM7TS2DSPXA, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 188.166.225.170:41538->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265359680 (187h35m47.945555239s) [noise=true , address=77.90.31.10:4518 , piece_id=FSK2TVCFOZPAVVZ3FICC72PBE5UXR2ACRDE4YZAMM5GVSGCIU53Q, seg_pos={0 0}, upload=165063]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 188.166.225.170:46784->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:51.265374649 (187h35m47.945570208s) [seg_pos={0 0}, upload=165063]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 31.509814372s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [..................................X.X.X.X.XX.XXX.X..X...X..X] duration: 30.00061459s FAIL (2025-09-06 09:43:50 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.000642363s core uplink us1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 1.871763391s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:43:43 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 1.8717843s edge linksharing us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration: 30.00059262s FAIL (2025-09-06 09:43:39 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 30.000617126s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000403897s FAIL (2025-09-06 09:43:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000413608s edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 3.3523286s FAIL (2025-09-06 09:43:34 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.352354671s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXXX...XXXXXX...X.] duration: 29.462954216s FAIL (2025-09-06 09:43:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 29.462986068s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 1m0.180141635s FAIL (2025-09-06 09:43:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 4) ======LOGS====== e=true , address=185.167.204.206:4055 , piece_id=SHZI4RHUW3RUCUQJ5PWKHMBRP4FWROE2T77HDOF5OSBGWQXAIWIA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12CYRieezm1mjtGchWdPrB2mNaHVRvTS85umjB6cQyYKnvisXSB, address:185.167.204.206:4055): protocol: write tcp 188.166.225.170:56640->185.167.204.206:4055: use of closed network connection; write tcp 188.166.225.170:56640->185.167.204.206:4055: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56640->185.167.204.206:4055: use of closed network connection Sep 6 09:43:26.907881844 (187h35m23.588077413s) [noise=true , address=185.167.204.206:4065 , piece_id=K73BBGSFLQM4YPH3CRMALVIA3ULOB7G7WUYO75JZMSSWSTYXCEIQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12oPo26KZGZr7o7M18KZiX5PMAAMwHK8FPT59y1hSLhkB6z3Lcj, address:185.167.204.206:4065): protocol: read tcp 188.166.225.170:48156->185.167.204.206:4065: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:26.907913073 (187h35m23.588108642s) [noise=true , address=185.167.204.205:4026 , piece_id=MIBJOI6GHJ7KRDD5DJPV6WXZEWMJVWQ2UHWAQFMN4YSCTV6MIACQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: write tcp 188.166.225.170:41670->185.167.204.205:4026: use of closed network connection; write tcp 188.166.225.170:41670->185.167.204.205:4026: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41670->185.167.204.205:4026: use of closed network connection Sep 6 09:43:26.907955775 (187h35m23.588151344s) [noise=true , address=185.167.204.205:4021 , piece_id=HO5OKT22LY6QAV7WB64AALEGD7UNSHSFDIWV65WDFCKZVC5723KA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, address:185.167.204.205:4021): protocol: write tcp 188.166.225.170:55462->185.167.204.205:4021: use of closed network connection; write tcp 188.166.225.170:55462->185.167.204.205:4021: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55462->185.167.204.205:4021: use of closed network connection Sep 6 09:43:26.908045256 (187h35m23.588240825s) [noise=true , address=81.171.14.70:4035 , piece_id=CI2X3VKDZJYA5K3NE67N2HNSQDCAAZSMTWSUA3MCPNDI6SW7KSMA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: write tcp 188.166.225.170:33306->81.171.14.70:4035: use of closed network connection; write tcp 188.166.225.170:33306->81.171.14.70:4035: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33306->81.171.14.70:4035: use of closed network connection Sep 6 09:43:26.908065235 (187h35m23.588260804s) [noise=true , address=185.167.204.205:4028 , piece_id=5MALGRNOJZJVVK6R5CV4WLOZSLOWTKNDVVYICJ7GODQ4YUAN7AUA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1HZoGwBuBh9Kg7gkvH9pfrjtxQurwxdnF7pWfycGuLHrHTSq61, address:185.167.204.205:4028): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:43:26.908102406 (187h35m23.588297965s) [noise=true , address=185.167.204.206:4046 , piece_id=Z4DTBGHCKGQ6KOFCRPWSHLS5PMCB3O42ETWZHUNJ6MMI72DPBLVA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: read tcp 188.166.225.170:48112->185.167.204.206:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:26.908127233 (187h35m23.588322802s) [noise=true , address=81.171.14.70:4037 , piece_id=56FXXA637MII5P6B4KLYMM3HVKSCABKAE3UVT2BFZWNS6JCDSMCA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: write tcp 188.166.225.170:51404->81.171.14.70:4037: use of closed network connection; write tcp 188.166.225.170:51404->81.171.14.70:4037: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51404->81.171.14.70:4037: use of closed network connection Sep 6 09:43:26.908158914 (187h35m23.588354483s) [noise=true , address=185.167.204.206:4060 , piece_id=2WO33TGMH27SVYYMHCQH2Y7SRELIYC4YHEM66IRLKVB35EN22REA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12RJC6wKVpcVGVNxVSXEYVXCVhC1vDPjbduZ1cFmCDPaYf1WTSr, address:185.167.204.206:4060): protocol: read tcp 188.166.225.170:41712->185.167.204.206:4060: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:26.908180285 (187h35m23.588375844s) [noise=true , address=81.171.14.70:4033 , piece_id=PCOMREMUBHUVYSBPIBYM3NUFMGQR3ZSWCON6CGA6ICMFFRRASMMQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:44322->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:44322->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44322->81.171.14.70:4033: use of closed network connection Sep 6 09:43:26.908187498 (187h35m23.588383067s) [noise=true , address=81.171.14.70:4032 , piece_id=XQQGOP7LYXZJBSL3XMG4KRLBPH27RT24F4SV2A3RYOVHUYPMA35Q, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:37550->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:37550->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37550->81.171.14.70:4032: use of closed network connection Sep 6 09:43:26.908196405 (187h35m23.588391974s) [noise=true , address=185.167.204.206:4061 , piece_id=HLEVZGC3RLM4O7HKAE4L4IMX6LWUGE6QC5NB46T4KMNILMZMAXSA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1G18tJ6BsYF3A1sLpRTTGzomqRWsRsj8tbHMJQeR1tZunoTDia, address:185.167.204.206:4061): protocol: write tcp 188.166.225.170:47874->185.167.204.206:4061: use of closed network connection; write tcp 188.166.225.170:47874->185.167.204.206:4061: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47874->185.167.204.206:4061: use of closed network connection Sep 6 09:43:26.908206315 (187h35m23.588401884s) [noise=true , address=185.167.204.205:4013 , piece_id=OE4HRDIYHMCZ6AR5MXDDPCCJGIRKUTH7ECDKU5PJYTU22SFLX3LA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1DJ9zLuqKQthmHgaSqovGQo8Sq4h8ymNV3vYc5XapvsxCVBrL5, address:185.167.204.205:4013): protocol: write tcp 188.166.225.170:53550->185.167.204.205:4013: use of closed network connection; write tcp 188.166.225.170:53550->185.167.204.205:4013: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53550->185.167.204.205:4013: use of closed network connection Sep 6 09:43:26.908222445 (187h35m23.588418014s) [noise=true , address=81.171.14.70:4047 , piece_id=6BJV5RNR3T6X6IQRVOIDRYGVZAVD5CUO4556QMVATQAKAHGCNNVA, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:43:26.908245139 (187h35m23.588440708s) [noise=true , address=185.167.204.206:4069 , piece_id=XV23AXFLUU5VHULRVKWXIS52YA5J6CZGIO55CJXGTCYUMCMNY6TQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12Uan1TT5oAM3axhaviVFbvh83gThiZvwHMwbCD31ZZens6BQ4U, address:185.167.204.206:4069): protocol: write tcp 188.166.225.170:53914->185.167.204.206:4069: use of closed network connection; write tcp 188.166.225.170:53914->185.167.204.206:4069: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53914->185.167.204.206:4069: use of closed network connection Sep 6 09:43:26.908270106 (187h35m23.588465665s) [noise=true , address=185.167.204.205:4012 , piece_id=4BMRVFZMBXWWTCIFN33GZEG7MTT56VXKJFYBQNGIIXEOIZB26HUQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: write tcp 188.166.225.170:39200->185.167.204.205:4012: use of closed network connection; write tcp 188.166.225.170:39200->185.167.204.205:4012: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39200->185.167.204.205:4012: use of closed network connection Sep 6 09:43:26.908295225 (187h35m23.588490794s) [noise=true , address=185.167.204.205:4033 , piece_id=H5ERZFAX2AGND7MMQZXNP6U6QEAQOZBH2YAPCTQ6CAYCB3PRAELQ, seg_pos={0 0}, upload=165052]: Done uploading piece. err: ecclient: upload failed (node:12Q73mXHLEfuX4VUbJ8PTjhMoXCrYSKjJbFxqjDdqF3U99N8FXe, address:185.167.204.205:4033): protocol: read tcp 188.166.225.170:43982->185.167.204.205:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:26.908312408 (187h35m23.588507977s) [seg_pos={0 0}, upload=165052]: Done waiting for segment. successful: 4 optimal: 26 errs: <nil> Sep 6 09:43:27.085351459 (187h35m23.765547028s) [upload=165052]: Done uploading segments... Sep 6 09:43:27.085381546 (187h35m23.765577115s) [upload=165052]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 4) Duration: 1m0.180159478s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration: 31.533028295s FAIL (2025-09-06 09:43:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== 027048943s) [noise=true , address=77.90.31.15:4699 , piece_id=CK3VWO7P456R2HY3G2MG4M7C4WWDEJRVQAJZJCWTDXY5QVZLXZ5A, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.346912948 (187h34m58.027108517s) [noise=true , address=77.90.31.19:4843 , piece_id=WE4VCSVEBFXVVIJMRDFO4XUWPDRZAQ7TRYLSKI5IAVCH5O5DOL7Q, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347136756 (187h34m58.027332325s) [noise=true , address=77.90.31.15:4698 , piece_id=TQEUJIE6R4I56SAAALNSDTRAUEYHOD7YSSSESAGW4M2WYBOXYAOQ, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347156634 (187h34m58.027352203s) [noise=true , address=45.91.116.11:4117 , piece_id=7QDSM4TBJZ52JAN4TAUTN2534XH6IPHEZKEEG6CGZRZDG2MC2BYA, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347355655 (187h34m58.027551254s) [noise=true , address=45.91.116.4:4938 , piece_id=ZVJHWSYEPXYKTUZECOCEHGTAUNBM5YOMBV54RO3WM4IT6UQUW3PA, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347467058 (187h34m58.027662627s) [noise=true , address=77.90.31.18:4806 , piece_id=57XHMWQDESZ6ELTNQZ2AJMD5442DQMJYCYYD5HFXVXAPNKIC5AQQ, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347707348 (187h34m58.027902917s) [noise=true , address=45.91.116.9:4262 , piece_id=ZB4L4P35ATE35CXOLC3ZQ67W3J2WC27CDCJA2VOGV3XA2XWSQX4A, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:01.347920326 (187h34m58.028115895s) [noise=true , address=45.91.116.8:4048 , piece_id=5DAKAWK4HLVSULDVOW2MYIYIAQ4VJ7OIJNIWL76AY3KPEN2JKDYQ, seg_pos={0 0}, upload=165055]: Uploading piece... Sep 6 09:43:19.482922493 (187h35m16.163118062s) [upload=165055]: Next segment err: context deadline exceeded Sep 6 09:43:19.483704500 (187h35m16.163900079s) [noise=true , address=45.91.116.5:4299 , piece_id=MCCF43OJLUYMCNC5T6MKWPSNZTJHWAZHFC7TNRWIT4FBUARFCZOQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: write tcp 188.166.225.170:38994->45.91.116.5:4299: use of closed network connection; write tcp 188.166.225.170:38994->45.91.116.5:4299: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38994->45.91.116.5:4299: use of closed network connection Sep 6 09:43:19.483826363 (187h35m16.164021932s) [noise=true , address=77.90.31.19:4843 , piece_id=WE4VCSVEBFXVVIJMRDFO4XUWPDRZAQ7TRYLSKI5IAVCH5O5DOL7Q, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:57630->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:57630->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57630->77.90.31.19:4843: use of closed network connection Sep 6 09:43:19.484214136 (187h35m16.164409705s) [noise=true , address=45.91.116.13:4154 , piece_id=SHISDRWJSLNK63RSXFMZ7ZIV24KQ6MM4MF2BPDWMFLHALDHTWRAA, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 188.166.225.170:47026->45.91.116.13:4154: use of closed network connection; write tcp 188.166.225.170:47026->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47026->45.91.116.13:4154: use of closed network connection Sep 6 09:43:19.484271646 (187h35m16.164467215s) [noise=true , address=77.90.31.10:4517 , piece_id=WGDF6OUDNQLA4XHAGZ4ZUBC4RWAXD3SKOXWBI52WURMXVL46ECFA, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: read tcp 188.166.225.170:55078->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484321090 (187h35m16.164516659s) [noise=true , address=77.90.31.15:4699 , piece_id=CK3VWO7P456R2HY3G2MG4M7C4WWDEJRVQAJZJCWTDXY5QVZLXZ5A, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: read tcp 188.166.225.170:46632->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484393679 (187h35m16.164589248s) [noise=true , address=45.91.116.11:4117 , piece_id=7QDSM4TBJZ52JAN4TAUTN2534XH6IPHEZKEEG6CGZRZDG2MC2BYA, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:59192->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484431812 (187h35m16.164627381s) [noise=true , address=45.91.116.7:4009 , piece_id=XWSGM4B2LX5FETYLM45S7X3PLCVAF3QRKBN276O3DO35SFDOR6YQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: read tcp 188.166.225.170:42680->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484466027 (187h35m16.164661596s) [noise=true , address=77.90.31.12:4592 , piece_id=ADSQQ4KKLTUPLSGXHYIMABDPF6HPCHCQW7EC6YGFGKOMLGN35ELA, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 188.166.225.170:56046->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484469344 (187h35m16.164664913s) [noise=true , address=77.90.31.18:4806 , piece_id=57XHMWQDESZ6ELTNQZ2AJMD5442DQMJYCYYD5HFXVXAPNKIC5AQQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: read tcp 188.166.225.170:41492->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484497477 (187h35m16.164693046s) [noise=true , address=45.91.116.9:4262 , piece_id=ZB4L4P35ATE35CXOLC3ZQ67W3J2WC27CDCJA2VOGV3XA2XWSQX4A, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 188.166.225.170:47726->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484530380 (187h35m16.164725949s) [noise=true , address=45.91.116.8:4048 , piece_id=5DAKAWK4HLVSULDVOW2MYIYIAQ4VJ7OIJNIWL76AY3KPEN2JKDYQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 188.166.225.170:53710->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484559957 (187h35m16.164755526s) [noise=true , address=45.91.116.14:4191 , piece_id=ZLHEFCMXHGS75R5JEHNL6Y4BW6ZXFKTSN6UMPY3NR4S525NOVCMQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 188.166.225.170:55988->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484590154 (187h35m16.164785723s) [noise=true , address=45.91.116.14:4189 , piece_id=T2QXQKAVWABHLVIOI6EEE2M4FAVIRYRQ7ZKQJDPCOR7WF4UR3XEQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 188.166.225.170:52430->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484595465 (187h35m16.164791034s) [noise=true , address=77.90.31.15:4698 , piece_id=TQEUJIE6R4I56SAAALNSDTRAUEYHOD7YSSSESAGW4M2WYBOXYAOQ, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: read tcp 188.166.225.170:56478->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484615593 (187h35m16.164811162s) [noise=true , address=45.91.116.4:4938 , piece_id=ZVJHWSYEPXYKTUZECOCEHGTAUNBM5YOMBV54RO3WM4IT6UQUW3PA, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:59152->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484656372 (187h35m16.164851941s) [noise=true , address=45.91.116.9:4263 , piece_id=5WNGHA6LB6LGQPWXV4Z4WPCYHZJRSMGMRRHEDFSI6P2BFUNSO45A, seg_pos={0 0}, upload=165055]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 188.166.225.170:41116->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:43:19.484672652 (187h35m16.164868211s) [seg_pos={0 0}, upload=165055]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 31.533051851s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [..........................................XX...XX....XXXX..X] duration: 1m0.000722379s FAIL (2025-09-06 09:43:12 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:42:20.441545931 (187h34m17.1217415s) [upload=165049]: Starting upload Sep 6 09:42:20.441596276 (187h34m17.121791845s) [upload=165049]: Done starting upload Sep 6 09:42:20.441666902 (187h34m17.121862471s) [upload=165049]: Uploading segments... Sep 6 09:42:20.441719463 (187h34m17.121915062s) [upload=165049]: Got next segment. Inline: true Sep 6 09:42:20.441725734 (187h34m17.121921303s) [upload=165049]: Waiting for error group managing segments... Sep 6 09:42:20.441774748 (187h34m17.121970317s) [upload=165049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:42:20.441780459 (187h34m17.121976028s) [upload=165049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:42:20.441784406 (187h34m17.121979976s) [upload=165049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:42:46.586524206 (187h34m43.266719775s) [upload=165049]: Done uploading segments... Sep 6 09:42:46.586540978 (187h34m43.266736547s) [upload=165049]: Upload finished. err: <nil> Duration:* 1m0.00073349s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 8.843430257s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:43:11 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 8.843472126s edge linksharing us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [................................................X.X..X.XX...] duration: 15.329641769s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:43:07 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 15.329667238s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.001357702s FAIL (2025-09-06 09:43:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001382449s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 1m0.00051594s FAIL (2025-09-06 09:43:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000526591s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXXX...XXXXXX...X] duration: 30.001579986s FAIL (2025-09-06 09:43:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001602989s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................X..X....X.X........X.X....] duration: 5.760553441s FAIL (2025-09-06 09:43:00 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 43) Duration: 5.760581304s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 1m0.476600141s FAIL (2025-09-06 09:42:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== ction; write tcp 188.166.225.170:58798->145.79.227.31:4802: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58798->145.79.227.31:4802: use of closed network connection Sep 6 09:42:53.822171956 (187h34m50.502367525s) [noise=true , address=145.79.227.12:4771 , piece_id=NM2VZXDAFT2VNPAWVU2Z7LOJBDRUKVEG3PKF3FHW5S7OJPK65NVQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1rcgrMxg2fRfvh7EA1eUehzD5rSH1krUd5W5eC8KTo7vjYWFLL, address:145.79.227.12:4771): protocol: write tcp 188.166.225.170:46908->145.79.227.12:4771: use of closed network connection; write tcp 188.166.225.170:46908->145.79.227.12:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46908->145.79.227.12:4771: use of closed network connection Sep 6 09:42:53.822211232 (187h34m50.502406801s) [noise=true , address=145.79.227.14:4861 , piece_id=FUQQ4U63PISBMGP4WA65VCE2VLAZFZLGPUKZEC7N7QZMXXJKS7WA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1SqXobRvE3NHfzYxdJ8sedJq3W9pQ6KpMLMiojUhpsCigeeNuS, address:145.79.227.14:4861): protocol: write tcp 188.166.225.170:60178->145.79.227.14:4861: use of closed network connection; write tcp 188.166.225.170:60178->145.79.227.14:4861: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60178->145.79.227.14:4861: use of closed network connection Sep 6 09:42:53.822302847 (187h34m50.502498416s) [noise=true , address=145.79.227.2:4119 , piece_id=UBX74NIPIDPJQ6MFNEIHHZXZ4QNAAMGRKNDLTXUWICZCF4GP67PA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:121xNAnkm9HZXUszp26UmWTsuhp3igyeDVea6gSCdtrQUQDVHpW, address:145.79.227.2:4119): protocol: write tcp 188.166.225.170:41784->145.79.227.2:4119: use of closed network connection; write tcp 188.166.225.170:41784->145.79.227.2:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41784->145.79.227.2:4119: use of closed network connection Sep 6 09:42:53.822346331 (187h34m50.5025419s) [noise=true , address=77.90.31.14:4689 , piece_id=QVO6ZP2KYOFBHNZ6Q7XC3TKHUZPL6TI2OLSU75OBF6XCF2PNF3YA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: write tcp 188.166.225.170:39350->77.90.31.14:4689: use of closed network connection; write tcp 188.166.225.170:39350->77.90.31.14:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39350->77.90.31.14:4689: use of closed network connection Sep 6 09:42:53.822455319 (187h34m50.502650888s) [noise=true , address=45.91.116.5:4305 , piece_id=VUVSKWHMZKPC6WHQOHZAJUHUNBYTOSTZ65VGSPBAEZZXWTDVQVLA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, address:45.91.116.5:4305): protocol: write tcp 188.166.225.170:44060->45.91.116.5:4305: use of closed network connection; write tcp 188.166.225.170:44060->45.91.116.5:4305: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44060->45.91.116.5:4305: use of closed network connection Sep 6 09:42:53.822491318 (187h34m50.502686887s) [noise=true , address=45.91.116.7:4041 , piece_id=7ARPBFG3B4VLJWYGPIYADJVP26V47F4FPI2FPLJAFXA3Y7DH46OA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1BvWxeWEnE4X8CJA6KJKryewizvzdGnbmthSZmr2Rfb7LdtUHV, address:45.91.116.7:4041): protocol: write tcp 188.166.225.170:33128->45.91.116.7:4041: use of closed network connection; write tcp 188.166.225.170:33128->45.91.116.7:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33128->45.91.116.7:4041: use of closed network connection Sep 6 09:42:53.822527847 (187h34m50.502723416s) [noise=true , address=145.79.227.5:4239 , piece_id=ECK4KDRDV4F6PZXTUFXHW34SNVM5MDXYJWA6CX647VZ5TXTJDNRA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1pTRYQDUQVz3nNo46TfMeaihJoP2zvTMfgvTNjUMF8iePz5MvV, address:145.79.227.5:4239): protocol: write tcp 188.166.225.170:56210->145.79.227.5:4239: use of closed network connection; write tcp 188.166.225.170:56210->145.79.227.5:4239: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56210->145.79.227.5:4239: use of closed network connection Sep 6 09:42:53.822533399 (187h34m50.502728968s) [noise=true , address=145.79.227.33:4894 , piece_id=ZEZFU5N2P33SNGUVYOPY4GHATH2ZOJJCQEHIHUQVJVM6VHNNTRDA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44, address:145.79.227.33:4894): protocol: read tcp 188.166.225.170:42658->145.79.227.33:4894: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:53.822564508 (187h34m50.502760077s) [noise=true , address=77.90.31.13:4656 , piece_id=RYCYGMUCEMA423Y2ZPGV2NIDMZUOGZFJBXUDBGQNZXJJWATELX2Q, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN, address:77.90.31.13:4656): protocol: write tcp 188.166.225.170:33856->77.90.31.13:4656: use of closed network connection; write tcp 188.166.225.170:33856->77.90.31.13:4656: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->77.90.31.13:4656: use of closed network connection Sep 6 09:42:53.822589716 (187h34m50.502785286s) [noise=true , address=77.90.31.17:4803 , piece_id=JXY3W7FDXUA6LCERBGSFUWHGBM5WVAFSRUGQLIMFNLJNS2TPAFEQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12onacKUY32faZTFuyEjvMCZA4nVKiRKmTKh1WRFMfif2Kw1Q7Y, address:77.90.31.17:4803): protocol: write tcp 188.166.225.170:49404->77.90.31.17:4803: use of closed network connection; write tcp 188.166.225.170:49404->77.90.31.17:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49404->77.90.31.17:4803: use of closed network connection Sep 6 09:42:53.822610676 (187h34m50.502806246s) [noise=true , address=145.79.227.18:4030 , piece_id=NZDORDLQ6EX2YBOWN3OSFVTZAXZ2DBPKVVJLYWMX3MXHVQQMQYQQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: write tcp 188.166.225.170:39562->145.79.227.18:4030: use of closed network connection; write tcp 188.166.225.170:39562->145.79.227.18:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39562->145.79.227.18:4030: use of closed network connection Sep 6 09:42:53.822631426 (187h34m50.502826995s) [noise=true , address=145.79.227.26:4661 , piece_id=7NE3HMCAW3SNPOO2JAYWZJVAVVX3TNETSCFLLJBBPGNRVVMKZTGQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12jxHbnubVkjisrfmNbLTpYRbBpr5jEyJSMSVUcj57pb1sCrG4i, address:145.79.227.26:4661): protocol: write tcp 188.166.225.170:57468->145.79.227.26:4661: use of closed network connection; write tcp 188.166.225.170:57468->145.79.227.26:4661: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57468->145.79.227.26:4661: use of closed network connection Sep 6 09:42:53.822669740 (187h34m50.502865309s) [noise=true , address=77.90.31.21:4907 , piece_id=XZDR2AQ44XJ3WU36BA372MN5VIHN5TATMJGBGLU6UXQGO52BLOPQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12RCGnjZC1UT9SUmFirNhS9pRBtxjcy91ss7c3Mk2kKnnHn2F6s, address:77.90.31.21:4907): protocol: write tcp 188.166.225.170:35858->77.90.31.21:4907: use of closed network connection; write tcp 188.166.225.170:35858->77.90.31.21:4907: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35858->77.90.31.21:4907: use of closed network connection Sep 6 09:42:53.822688775 (187h34m50.502884344s) [noise=true , address=145.79.227.37:4579 , piece_id=I5YP5WLKVCJCHTMKHMYVXN3VOZS4I23H7PEBOXHCFJTMRNUUTWRQ, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12ZMdnhz2eNgqaF2DpsJmyb6xJQKFYvmitNy6fqMugcnsuZqmak, address:145.79.227.37:4579): protocol: write tcp 188.166.225.170:48276->145.79.227.37:4579: use of closed network connection; write tcp 188.166.225.170:48276->145.79.227.37:4579: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48276->145.79.227.37:4579: use of closed network connection Sep 6 09:42:53.822711990 (187h34m50.502907559s) [noise=true , address=145.79.227.19:4098 , piece_id=WTCHUWKWAJRURJDPMFOFGHN6VF453PKI3CVQ23WHTPCHW7TDSLYA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:122fmWY53WxM6KHuXzdKsCbWWCtCFDUGowvSqALWUdFnbYVJoSY, address:145.79.227.19:4098): protocol: write tcp 188.166.225.170:37062->145.79.227.19:4098: use of closed network connection; write tcp 188.166.225.170:37062->145.79.227.19:4098: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37062->145.79.227.19:4098: use of closed network connection Sep 6 09:42:53.822778949 (187h34m50.502974518s) [noise=true , address=77.90.31.15:4706 , piece_id=DC56DODIYN5ELFCJGVSJQ5KN2Y2XDNJMHBKIL3QRDJQHVYXSQ5KA, seg_pos={0 0}, upload=165044]: Done uploading piece. err: ecclient: upload failed (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG, address:77.90.31.15:4706): protocol: read tcp 188.166.225.170:43908->77.90.31.15:4706: use of closed network connection; context deadline exceeded Sep 6 09:42:53.822794588 (187h34m50.502990157s) [seg_pos={0 0}, upload=165044]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m0.47661531s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 7 out of 60 is failed (threshold=3) [.....................................XXX....X.XX..X.........] duration: 10.678656928s target: https://us1.storj.io/login FAIL (2025-09-06 09:42:51 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 10.678684022s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [..................................XXXXXX.....XXX.X.X.XXXX...] duration: 10.137284035s FAIL (2025-09-06 09:42:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:40:55.687477176 (187h32m52.367672745s) [upload=165027]: Starting upload Sep 6 09:40:55.687529276 (187h32m52.367724835s) [upload=165027]: Done starting upload Sep 6 09:40:55.687623947 (187h32m52.367819516s) [upload=165027]: Uploading segments... Sep 6 09:40:55.687661369 (187h32m52.367856938s) [upload=165027]: Got next segment. Inline: true Sep 6 09:40:55.687665246 (187h32m52.367860815s) [upload=165027]: Waiting for error group managing segments... Sep 6 09:40:55.687720752 (187h32m52.367916321s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:40:55.687725201 (187h32m52.36792077s) [upload=165027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:40:55.687728007 (187h32m52.367923576s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:41:17.325212667 (187h33m14.005408236s) [upload=165027]: Done uploading segments... Sep 6 09:41:17.325231412 (187h33m14.005426981s) [upload=165027]: Upload finished. err: <nil> Duration:* 10.137312239s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [...................................X.X.X.X.XX.XXX.X..X...X..] duration: 22.464123992s FAIL (2025-09-06 09:42:36 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 22.464149201s core uplink us1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 5 out of 60 is failed (threshold=3) [...........................................X..X..X..X.X.....] duration: 22.148133062s FAIL (2025-09-06 09:42:36 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 22.148162647s edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration: 30.463679843s FAIL (2025-09-06 09:42:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== upload=165045]: Uploading piece... Sep 6 09:42:10.956130672 (187h34m7.636326241s) [noise=true , address=45.91.116.14:4189 , piece_id=HFAURDNOACDNNJ2CV5B7FMFUHIQHLTBL6FWQBODG5DTF5VONDJZQ, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.956310667 (187h34m7.636506236s) [noise=true , address=45.91.116.8:4047 , piece_id=SV4YQXFBJ2MJX27JT7WRHXRZ3IKQUJWTGXACLNDSDDKDUWAEA3BA, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.956358469 (187h34m7.636554038s) [noise=true , address=45.91.116.4:4940 , piece_id=LHDMDHO6J3VST5XDGELSPU3QSPVUW5UEEADKGKO3YPBW4QD55C7A, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.956603858 (187h34m7.636799427s) [noise=true , address=45.91.116.10:4083 , piece_id=HMD763UPRFE4H64AZMGIR7G5F6XLQYEL4JFQETEZ2UJB7T7YP7YQ, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.956660457 (187h34m7.636856026s) [noise=true , address=45.91.116.13:4154 , piece_id=RVMIGRA4J5IV6S6GCJREOYFDXVWODOQ3QX7KC4GBU7W4C4PJX7LA, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.956955872 (187h34m7.637151441s) [noise=true , address=77.90.31.14:4661 , piece_id=C4TVITH43FU6O7SRSXSCM3I7Y3J7G7V7TG5DKNBFEUBDRQ3LGWIQ, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:10.957288288 (187h34m7.637483857s) [noise=true , address=45.91.116.6:4973 , piece_id=YWEDY7DIZ754NYAFUM53CLFCU4MFVELWPWBPUXM2O66ZRVUJCQCA, seg_pos={0 0}, upload=165045]: Uploading piece... Sep 6 09:42:29.943463329 (187h34m26.623658898s) [upload=165045]: Next segment err: context deadline exceeded Sep 6 09:42:29.943884566 (187h34m26.624080135s) [noise=true , address=45.91.116.5:4298 , piece_id=IUG5ZHUT2QXNWX5YVHI6MGXLUDBY4HDYOMHPXLVC5GO5JZFUWZFQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 188.166.225.170:50682->45.91.116.5:4298: use of closed network connection; write tcp 188.166.225.170:50682->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50682->45.91.116.5:4298: use of closed network connection Sep 6 09:42:29.944009534 (187h34m26.624205093s) [noise=true , address=77.90.31.18:4805 , piece_id=ETFFTG7HWKTBRHQM4NOPGKZAGMLGT2P2HVUP6F5PL6G5ZJABCNHQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 188.166.225.170:41148->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944128993 (187h34m26.624324562s) [noise=true , address=45.91.116.11:4118 , piece_id=FUX7HTMWSUOJFQY4QROY4V3EI4LO7CZMC4UHGZBEINSOVA2GUCJQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p, address:45.91.116.11:4118): protocol: write tcp 188.166.225.170:32790->45.91.116.11:4118: use of closed network connection; write tcp 188.166.225.170:32790->45.91.116.11:4118: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32790->45.91.116.11:4118: use of closed network connection Sep 6 09:42:29.944233263 (187h34m26.624428832s) [noise=true , address=45.91.116.14:4189 , piece_id=HFAURDNOACDNNJ2CV5B7FMFUHIQHLTBL6FWQBODG5DTF5VONDJZQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: write tcp 188.166.225.170:55058->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:55058->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55058->45.91.116.14:4189: use of closed network connection Sep 6 09:42:29.944245787 (187h34m26.624441356s) [noise=true , address=45.91.116.10:4083 , piece_id=HMD763UPRFE4H64AZMGIR7G5F6XLQYEL4JFQETEZ2UJB7T7YP7YQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:43408->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944278259 (187h34m26.624473828s) [noise=true , address=45.91.116.4:4940 , piece_id=LHDMDHO6J3VST5XDGELSPU3QSPVUW5UEEADKGKO3YPBW4QD55C7A, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: write tcp 188.166.225.170:57914->45.91.116.4:4940: use of closed network connection; write tcp 188.166.225.170:57914->45.91.116.4:4940: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57914->45.91.116.4:4940: use of closed network connection Sep 6 09:42:29.944283679 (187h34m26.624479248s) [noise=true , address=77.90.31.10:4519 , piece_id=HSMVOMO2MSIAOS4H45SJ7GC54R3BTUNGHWC3ZI2TF24GQQ2ZTOHQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: read tcp 188.166.225.170:50332->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944328485 (187h34m26.624524054s) [noise=true , address=45.91.116.8:4045 , piece_id=DCOXVDDQXTQXASFLIV2GPZIXGWV4JXWGDMAEBV25F7QC4F2UXWOQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 188.166.225.170:36926->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944358242 (187h34m26.624553811s) [noise=true , address=45.91.116.12:4227 , piece_id=IUPO2XA6DYB75V6WDKORF32NUQIFBVGS67BOP2NITHGIJTUKDZYA, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 188.166.225.170:58458->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944374143 (187h34m26.624569712s) [noise=true , address=45.91.116.6:4973 , piece_id=YWEDY7DIZ754NYAFUM53CLFCU4MFVELWPWBPUXM2O66ZRVUJCQCA, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:58834->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944404631 (187h34m26.6246002s) [noise=true , address=77.90.31.19:4841 , piece_id=IHWTFE2W5N2OUR2FM3TXMP3KSKH2ONXDQFTVTIJKVLHSXMYRS7UQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 188.166.225.170:60014->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944418657 (187h34m26.624614216s) [noise=true , address=45.91.116.13:4154 , piece_id=RVMIGRA4J5IV6S6GCJREOYFDXVWODOQ3QX7KC4GBU7W4C4PJX7LA, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 188.166.225.170:37106->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944436642 (187h34m26.624632211s) [noise=true , address=45.91.116.8:4047 , piece_id=SV4YQXFBJ2MJX27JT7WRHXRZ3IKQUJWTGXACLNDSDDKDUWAEA3BA, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: read tcp 188.166.225.170:45656->45.91.116.8:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944454136 (187h34m26.624649705s) [noise=true , address=45.91.116.4:4939 , piece_id=7WHCVPJWEEC6NI6G5T4QONAKOL4GTEVICHJMXI6O5SKLQ72TOCCQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 188.166.225.170:36724->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944478502 (187h34m26.624674061s) [noise=true , address=45.91.116.6:4974 , piece_id=GSIRVFINMPEKXXWHBUEMN7OWFIDSITANORFVOKHT7IXM76WBNNUA, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: read tcp 188.166.225.170:33514->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944486998 (187h34m26.624682567s) [noise=true , address=77.90.31.14:4661 , piece_id=C4TVITH43FU6O7SRSXSCM3I7Y3J7G7V7TG5DKNBFEUBDRQ3LGWIQ, seg_pos={0 0}, upload=165045]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 188.166.225.170:35278->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:29.944500894 (187h34m26.624696463s) [seg_pos={0 0}, upload=165045]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 30.46370505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.003786046s FAIL (2025-09-06 09:42:29 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.003809532s core uplink us1
checkers.HTTP{"GET" "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt", ExpectedStatus: 308, ExpectedRedirect: "https://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt"} failure (antiblip window 3/60): (http.us_select_1.linksharing) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 24.235310776s target: http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt FAIL (2025-09-06 09:42:28 UTC) orbiter warning: Get "http://link.us-select-1.storjshare.io/raw/jv6fvdi5muos6nw22tdksxgfrd2a/uptime-checks-us-select/hello-world.txt": dial tcp 45.38.24.2:80: i/o timeout Duration: 24.235340272s edge linksharing us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 1m0.197632121s FAIL (2025-09-06 09:42:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 9) ======LOGS====== address:81.171.14.70:4026): protocol: expected piece hash; context deadline exceeded; EOF Sep 6 09:42:26.478143226 (187h34m23.158338795s) [noise=true , address=185.167.204.206:4045 , piece_id=QUKVCBGRWAK3UKYYOMU3B5ATDAMYDLVKLZBENNUYRK4ICWWCROQA, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq, address:185.167.204.206:4045): protocol: write tcp 188.166.225.170:39168->185.167.204.206:4045: use of closed network connection; write tcp 188.166.225.170:39168->185.167.204.206:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39168->185.167.204.206:4045: use of closed network connection Sep 6 09:42:26.478164517 (187h34m23.158360086s) [noise=true , address=81.171.14.70:4028 , piece_id=5PFJG6WWP5KH3N3O3A6RWQ77GQLWQE5AUDHAVGPN2N6QOC7OHQCQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: write tcp 188.166.225.170:42144->81.171.14.70:4028: use of closed network connection; write tcp 188.166.225.170:42144->81.171.14.70:4028: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42144->81.171.14.70:4028: use of closed network connection Sep 6 09:42:26.478295326 (187h34m23.158490915s) [noise=true , address=185.167.204.205:4032 , piece_id=VVQ5G3Z5LFBFTMDDSB2F33HTHJUREP7LEMFIE2VOCTODAVBARXEQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12558dx7zMNDfabV55vSsP9XwdxXgoPM8aHW53femVZzkCRxsKN, address:185.167.204.205:4032): protocol: write tcp 188.166.225.170:47532->185.167.204.205:4032: use of closed network connection; write tcp 188.166.225.170:47532->185.167.204.205:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47532->185.167.204.205:4032: use of closed network connection Sep 6 09:42:26.478347837 (187h34m23.158543406s) [noise=true , address=81.171.14.70:4025 , piece_id=R2I6TSIVZC7JECPSL6LLA4LPPZ2VJ7YJPGFJ3FLXECSJZR6HI5OQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: write tcp 188.166.225.170:33802->81.171.14.70:4025: use of closed network connection; write tcp 188.166.225.170:33802->81.171.14.70:4025: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33802->81.171.14.70:4025: use of closed network connection Sep 6 09:42:26.478418613 (187h34m23.158614182s) [noise=true , address=185.167.204.205:4017 , piece_id=CMJZVONX6JPU2AOMQV2XGNVYQEIWXH773KDPOAZQ7GMMZC45HP6Q, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 188.166.225.170:45864->185.167.204.205:4017: use of closed network connection; write tcp 188.166.225.170:45864->185.167.204.205:4017: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45864->185.167.204.205:4017: use of closed network connection Sep 6 09:42:26.478465763 (187h34m23.158661332s) [noise=true , address=185.167.204.206:4063 , piece_id=FQM3P6EACAXTJXLVHARIYYHGDDYPCGSVXLKNKSQ3QMNDQN77KWYA, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:16qB6oqhjMfxCSDZJers7bCt7Eeomd3njuz1qR5F9PTpuoHX5U, address:185.167.204.206:4063): protocol: write tcp 188.166.225.170:48974->185.167.204.206:4063: use of closed network connection; write tcp 188.166.225.170:48974->185.167.204.206:4063: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48974->185.167.204.206:4063: use of closed network connection Sep 6 09:42:26.478498195 (187h34m23.158693764s) [noise=true , address=81.171.14.70:4032 , piece_id=QYN2UDP6AHZHHZDYWLDG27Y4N5YJCEJ5AKSXBJXY2UTTWOHCW2EA, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1T7zKyoeXVwkgTKXkQqvXi28sWXHE9vRdcNYg7xR8tWFxdtDqf, address:81.171.14.70:4032): protocol: write tcp 188.166.225.170:51014->81.171.14.70:4032: use of closed network connection; write tcp 188.166.225.170:51014->81.171.14.70:4032: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51014->81.171.14.70:4032: use of closed network connection Sep 6 09:42:26.478506080 (187h34m23.158701649s) [noise=true , address=185.167.204.205:4036 , piece_id=B3YTTF3QPEVQ2LYVXV447B4SVVSXBF2P47P72LXQWSQTI5S6UVGQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12Mngc2aRWoo1ZQGktFfA4DauhKyEFcbRk6dg7nYa1jZUg2srMV, address:185.167.204.205:4036): protocol: write tcp 188.166.225.170:37132->185.167.204.205:4036: use of closed network connection; write tcp 188.166.225.170:37132->185.167.204.205:4036: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37132->185.167.204.205:4036: use of closed network connection Sep 6 09:42:26.478540676 (187h34m23.158736245s) [noise=true , address=185.167.204.205:4007 , piece_id=3FDC2KWOHLSGBGOSJQDDQ5IULOLM27G2WQNWA3VJBU76FL7EZGGQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12m7e6EGQBC1wvbRQ5uwG8wuHbgZeyXPdj5XDMi1kgo5woPgXR, address:185.167.204.205:4007): protocol: write tcp 188.166.225.170:51150->185.167.204.205:4007: use of closed network connection; write tcp 188.166.225.170:51150->185.167.204.205:4007: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51150->185.167.204.205:4007: use of closed network connection Sep 6 09:42:26.478568128 (187h34m23.158763697s) [noise=true , address=81.171.14.70:4033 , piece_id=RKHJ4JQE4OPCMUDSU6J2Z5B42N3EUM462D5MNM7AZCGFO24M5GLQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: write tcp 188.166.225.170:60752->81.171.14.70:4033: use of closed network connection; write tcp 188.166.225.170:60752->81.171.14.70:4033: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60752->81.171.14.70:4033: use of closed network connection Sep 6 09:42:26.478648743 (187h34m23.158844302s) [noise=true , address=185.167.204.205:4003 , piece_id=VDL4QM7P6XZC7RZBG3HOOMTCLEAIV7TEWCM4BPDDMASRV7GVJDBQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1CjWvvJ4FdZc41sRgsQFWz4Y1ATZKgzsvDcUTmBSrCC15ekCmF, address:185.167.204.205:4003): protocol: write tcp 188.166.225.170:34720->185.167.204.205:4003: use of closed network connection; write tcp 188.166.225.170:34720->185.167.204.205:4003: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34720->185.167.204.205:4003: use of closed network connection Sep 6 09:42:26.478654423 (187h34m23.158849992s) [noise=true , address=185.167.204.205:4020 , piece_id=I5UW3V6EBZQUZ5ME2SFS32DR2OFEYJRJWBMLFKEZ5WDMK3AEPXFQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: read tcp 188.166.225.170:48880->185.167.204.205:4020: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:26.478699690 (187h34m23.158895259s) [noise=true , address=185.167.204.205:4031 , piece_id=XZMS2AYLBNAVW43TIY6YYZGHEZHUZL3DFQTGB5I3CRVKLPLIJECQ, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1dBpSJ5HRaQRAayr7UmKuY8nDzJeh3MmgmduihjVYJGPUbRvkR, address:185.167.204.205:4031): protocol: read tcp 188.166.225.170:43670->185.167.204.205:4031: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:26.478712205 (187h34m23.158907784s) [noise=true , address=185.167.204.206:4050 , piece_id=EKN73NC5PFVXA2CZFPE6AQBJFWWRG335QQAMCZDDOWE4ZDABFOJA, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1UR3Amy4kaFW5xhpD1s4WAVW15smyz4up1EZ5SH5Eg126SnAaz, address:185.167.204.206:4050): protocol: write tcp 188.166.225.170:32820->185.167.204.206:4050: use of closed network connection; write tcp 188.166.225.170:32820->185.167.204.206:4050: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32820->185.167.204.206:4050: use of closed network connection Sep 6 09:42:26.478740399 (187h34m23.158935958s) [noise=true , address=185.167.204.205:4023 , piece_id=6H5T5TC5WR4ZQ3AQIITKDGN6VR5UU2SKMVM3IWXRI5SGEVJSUN4Q, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:12Mr4R5VQs9NbCHNZonSC42mesJNuJ2Co3BbU7LTfLPQwWEkQbo, address:185.167.204.205:4023): protocol: read tcp 188.166.225.170:33416->185.167.204.205:4023: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:26.478807737 (187h34m23.159003306s) [noise=true , address=185.167.204.205:4026 , piece_id=IAMIRGOTMR67RWBSNDZRBB5HIHQ5GHZ6SHMX7AANWRNAY74OIQBA, seg_pos={0 0}, upload=165035]: Done uploading piece. err: ecclient: upload failed (node:1A9VrfmePrZ6AiT4SgVgEEahfBc5u5e2LnQm86gEpEkQgqB2e4, address:185.167.204.205:4026): protocol: read tcp 188.166.225.170:53234->185.167.204.205:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:42:26.478838866 (187h34m23.159034435s) [seg_pos={0 0}, upload=165035]: Done waiting for segment. successful: 9 optimal: 26 errs: <nil> Sep 6 09:42:26.673096006 (187h34m23.353291575s) [upload=165035]: Done uploading segments... Sep 6 09:42:26.673122827 (187h34m23.353318396s) [upload=165035]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 9) Duration: 1m0.197644214s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [.................................................X.X..X.XX..] duration: 3.686976114s target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-09-06 09:42:25 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": dial tcp 45.38.24.2:443: i/o timeout Duration: 3.687002324s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 18 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXX...XXXXXX...] duration: 25.074401035s FAIL (2025-09-06 09:42:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 25.074423699s edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 3.473271219s FAIL (2025-09-06 09:42:20 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.473296106s core uplink ap1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 7 out of 60 is failed (threshold=3) [......................................XXX....X.XX..X........] duration: 4.385583143s target: https://us1.storj.io/login FAIL (2025-09-06 09:42:19 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: i/o timeout Duration: 4.385612398s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [...................................XXXXXX.....XXX.X.X.XXXX..] duration: 19.397640195s FAIL (2025-09-06 09:42:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:40:55.687477176 (187h32m52.367672745s) [upload=165027]: Starting upload Sep 6 09:40:55.687529276 (187h32m52.367724835s) [upload=165027]: Done starting upload Sep 6 09:40:55.687623947 (187h32m52.367819516s) [upload=165027]: Uploading segments... Sep 6 09:40:55.687661369 (187h32m52.367856938s) [upload=165027]: Got next segment. Inline: true Sep 6 09:40:55.687665246 (187h32m52.367860815s) [upload=165027]: Waiting for error group managing segments... Sep 6 09:40:55.687720752 (187h32m52.367916321s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:40:55.687725201 (187h32m52.36792077s) [upload=165027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:40:55.687728007 (187h32m52.367923576s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:41:17.325212667 (187h33m14.005408236s) [upload=165027]: Done uploading segments... Sep 6 09:41:17.325231412 (187h33m14.005426981s) [upload=165027]: Upload finished. err: <nil> Duration:* 19.397782417s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration: 30.000823546s FAIL (2025-09-06 09:42:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000848734s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [...........................................XX...XX....XXXX..] duration: 28.726744368s FAIL (2025-09-06 09:42:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:40:03.132906827 (187h31m59.813102396s) [upload=165017]: Starting upload Sep 6 09:40:03.132979777 (187h31m59.813175356s) [upload=165017]: Done starting upload Sep 6 09:40:03.133037107 (187h31m59.813232676s) [upload=165017]: Uploading segments... Sep 6 09:40:03.133078547 (187h31m59.813274126s) [upload=165017]: Got next segment. Inline: true Sep 6 09:40:03.133083235 (187h31m59.813278814s) [upload=165017]: Waiting for error group managing segments... Sep 6 09:40:03.133126047 (187h31m59.813321626s) [upload=165017]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:40:03.133131979 (187h31m59.813327558s) [upload=165017]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:40:03.133134964 (187h31m59.813330534s) [upload=165017]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:40:52.986772975 (187h32m49.666968544s) [upload=165017]: Done uploading segments... Sep 6 09:40:52.986797041 (187h32m49.66699261s) [upload=165017]: Upload finished. err: metaclient: context deadline exceeded Duration:* 28.72677257s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................X..X....X.X........X.X...] duration: 5.565302001s FAIL (2025-09-06 09:42:05 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 43) Duration: 5.565334562s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 1m0.00072651s FAIL (2025-09-06 09:42:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000741288s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 15 out of 60 is failed (threshold=3) [....................................XXXXXX.....XXX.X.X.XXXX.] duration: 30.000800703s FAIL (2025-09-06 09:41:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Sep 6 09:40:55.687477176 (187h32m52.367672745s) [upload=165027]: Starting upload Sep 6 09:40:55.687529276 (187h32m52.367724835s) [upload=165027]: Done starting upload Sep 6 09:40:55.687623947 (187h32m52.367819516s) [upload=165027]: Uploading segments... Sep 6 09:40:55.687661369 (187h32m52.367856938s) [upload=165027]: Got next segment. Inline: true Sep 6 09:40:55.687665246 (187h32m52.367860815s) [upload=165027]: Waiting for error group managing segments... Sep 6 09:40:55.687720752 (187h32m52.367916321s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 09:40:55.687725201 (187h32m52.36792077s) [upload=165027]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Sep 6 09:40:55.687728007 (187h32m52.367923576s) [upload=165027]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 09:41:17.325212667 (187h33m14.005408236s) [upload=165027]: Done uploading segments... Sep 6 09:41:17.325231412 (187h33m14.005426981s) [upload=165027]: Upload finished. err: <nil> Duration:* 30.000824117s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 31.486176298s FAIL (2025-09-06 09:41:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0) ======LOGS====== , upload=165034]: Uploading piece... Sep 6 09:41:32.060732764 (187h33m28.740928333s) [noise=true , address=45.91.116.12:4227 , piece_id=STFO2AX5AS5IP3RGJ27QWAPPPIM7W6QXNIPXGW3Y2WO5MGTUMSOQ, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.060788511 (187h33m28.74098408s) [noise=true , address=45.91.116.9:4262 , piece_id=POJL662HZCCLVXPTZLPOH6PPCDPMWM4VXIJ5MSYJ6NOHPRM7KLCA, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.061056473 (187h33m28.741252042s) [noise=true , address=77.90.31.14:4663 , piece_id=6AK5PEH2X345KX326L7OA3IDW4I3CTTFB4BBLKHJSRGHEBW6GTHQ, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.061106739 (187h33m28.741302298s) [noise=true , address=45.91.116.4:4940 , piece_id=CT6XSXB263HDM66TQXPOKGVNZERHVV4P7YE5GK2OUL6AE5VNBEJA, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.061381294 (187h33m28.741576853s) [noise=true , address=45.91.116.6:4973 , piece_id=X44L7O2HYT4W3DAO4EYQUYK7KIS3QM4K52O47GI2XE2UADNKPH2Q, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.061418566 (187h33m28.741614135s) [noise=true , address=45.91.116.4:4939 , piece_id=DVES6BLP55ZXBG26GH2ANELKLG5VQIBFRID2V35MBFYX6B5OPVNA, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:32.061640761 (187h33m28.74183633s) [noise=true , address=45.91.116.13:4154 , piece_id=NBBNHD3N25EHMN4Q2NTNPSESBYCBDJV6SFPTKJIGBBDIAGM2CMKQ, seg_pos={0 0}, upload=165034]: Uploading piece... Sep 6 09:41:52.779991305 (187h33m49.460186884s) [upload=165034]: Next segment err: context deadline exceeded Sep 6 09:41:52.780434002 (187h33m49.460629571s) [noise=true , address=45.91.116.8:4048 , piece_id=W5IIF7PLBJ764HUKOMUHMASFSXG77AGU6GMIFGM4FHRJGZ47S6ZA, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:58354->45.91.116.8:4048: use of closed network connection; write tcp 188.166.225.170:58354->45.91.116.8:4048: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58354->45.91.116.8:4048: use of closed network connection Sep 6 09:41:52.780546467 (187h33m49.460742036s) [noise=true , address=45.91.116.4:4939 , piece_id=DVES6BLP55ZXBG26GH2ANELKLG5VQIBFRID2V35MBFYX6B5OPVNA, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:33240->45.91.116.4:4939: use of closed network connection; write tcp 188.166.225.170:33240->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33240->45.91.116.4:4939: use of closed network connection Sep 6 09:41:52.780580483 (187h33m49.460776052s) [noise=true , address=45.91.116.6:4973 , piece_id=X44L7O2HYT4W3DAO4EYQUYK7KIS3QM4K52O47GI2XE2UADNKPH2Q, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 188.166.225.170:33564->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.780747943 (187h33m49.460943512s) [noise=true , address=77.90.31.12:4589 , piece_id=UYJF5ZXBYGLDGPNICUYNBCYGRKTPDP2LKQUHPAZF2ZY5XVFTMQ4Q, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:35236->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:35236->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35236->77.90.31.12:4589: use of closed network connection Sep 6 09:41:52.780962304 (187h33m49.461157873s) [noise=true , address=77.90.31.17:4770 , piece_id=HIBEPDYZOXCHF4RC3TCUPYRZO2XFRNRHJJATLODVYOOZ3E5ADFIQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:58260->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.780994245 (187h33m49.461189814s) [noise=true , address=45.91.116.10:4083 , piece_id=22DHNJ5H3IKK3H7G7DDUORHL5B37FRP73D5QNC2GJEIO2DBDJLCQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 188.166.225.170:38880->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781024382 (187h33m49.461219951s) [noise=true , address=77.90.31.10:4518 , piece_id=JHKQHRUNMEDR2HYVO2HKYLZUQNWDLXJ53ST6MJ5TDZI2AWCAMV5A, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 188.166.225.170:58978->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781054029 (187h33m49.461249598s) [noise=true , address=45.91.116.11:4117 , piece_id=56TX7QNVG2JMDPYGW5DMZMZWPCGSI2CEUJF45YKPA34275GNYV7Q, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 188.166.225.170:38010->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781082473 (187h33m49.461278032s) [noise=true , address=45.91.116.4:4940 , piece_id=CT6XSXB263HDM66TQXPOKGVNZERHVV4P7YE5GK2OUL6AE5VNBEJA, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: read tcp 188.166.225.170:49358->45.91.116.4:4940: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781118642 (187h33m49.461314211s) [noise=true , address=77.90.31.16:4735 , piece_id=3CG76TMTZWC37I4AV6UZNQIQTOID3B5NIZUAZ3TUKATDITKD2Y6A, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 188.166.225.170:35270->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781354313 (187h33m49.461549882s) [noise=true , address=45.91.116.9:4262 , piece_id=POJL662HZCCLVXPTZLPOH6PPCDPMWM4VXIJ5MSYJ6NOHPRM7KLCA, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:43484->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:43484->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43484->45.91.116.9:4262: use of closed network connection Sep 6 09:41:52.781442993 (187h33m49.461638562s) [noise=true , address=77.90.31.10:4519 , piece_id=IUNZT4EC6VNXK5MVLGAATNMBZ2WZ2A5NDZ2YDSRXQL7HISGAM5MQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: read tcp 188.166.225.170:45876->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781474984 (187h33m49.461670553s) [noise=true , address=45.91.116.4:4938 , piece_id=XMHFODDAJBAEIBYDXSD3GLKYBJ3PSMCDQXACG7DOWAFFN7XHGDYQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 188.166.225.170:35342->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781528276 (187h33m49.461723855s) [noise=true , address=45.91.116.12:4227 , piece_id=STFO2AX5AS5IP3RGJ27QWAPPPIM7W6QXNIPXGW3Y2WO5MGTUMSOQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: read tcp 188.166.225.170:52506->45.91.116.12:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781599903 (187h33m49.461795472s) [noise=true , address=77.90.31.14:4663 , piece_id=6AK5PEH2X345KX326L7OA3IDW4I3CTTFB4BBLKHJSRGHEBW6GTHQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 188.166.225.170:42254->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781777965 (187h33m49.461973534s) [noise=true , address=45.91.116.13:4154 , piece_id=NBBNHD3N25EHMN4Q2NTNPSESBYCBDJV6SFPTKJIGBBDIAGM2CMKQ, seg_pos={0 0}, upload=165034]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 188.166.225.170:51878->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:52.781838590 (187h33m49.462034179s) [seg_pos={0 0}, upload=165034]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil> Duration: 31.486201846s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.448737284s FAIL (2025-09-06 09:41:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: failed to upload enough pieces (needed at least 14 but got 0) ======LOGS====== 09:41:53.120103944 (187h33m49.800299513s) [noise=true , address=145.79.227.37:4589 , piece_id=LIU3MEOJUK64Z7PEN2EYLLOMMMO4MTSMI2PO42MPZLD63AJWGTUQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:12ChutvH6jC7eWE7dDE9Cmsn8EDo6ecHum8dG9NBvEcdphiJgbi, address:145.79.227.37:4589): protocol: write tcp 188.166.225.170:33842->145.79.227.37:4589: use of closed network connection; write tcp 188.166.225.170:33842->145.79.227.37:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33842->145.79.227.37:4589: use of closed network connection Sep 6 09:41:53.120256707 (187h33m49.800452276s) [noise=true , address=145.79.227.43:4174 , piece_id=W535CRBQBHTBLUM4BWT5ZOXBILQRSSO4Q3SZLN66YK764X3VROUA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1293SNDGTKNrYrKDuB5p2VpHWo7ANpQsVKVGa97jXVaGP83Pw51, address:145.79.227.43:4174): protocol: write tcp 188.166.225.170:37154->145.79.227.43:4174: use of closed network connection; write tcp 188.166.225.170:37154->145.79.227.43:4174: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37154->145.79.227.43:4174: use of closed network connection Sep 6 09:41:53.120272526 (187h33m49.800468095s) [noise=true , address=145.79.227.4:4188 , piece_id=DWQWVWX3SIUQZ622KEM2H4SYZXRA57I4S7ATXV75K7D4YZFBDPNQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: write tcp 188.166.225.170:36360->145.79.227.4:4188: use of closed network connection; write tcp 188.166.225.170:36360->145.79.227.4:4188: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36360->145.79.227.4:4188: use of closed network connection Sep 6 09:41:53.120353842 (187h33m49.800549411s) [noise=true , address=77.90.31.14:4689 , piece_id=J6TPLKMSXUGINLHEITF6UHJA5CRHZEA7BB6KMFFSICCVG4PJUFNA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: write tcp 188.166.225.170:35870->77.90.31.14:4689: use of closed network connection; write tcp 188.166.225.170:35870->77.90.31.14:4689: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35870->77.90.31.14:4689: use of closed network connection Sep 6 09:41:53.120397125 (187h33m49.800592694s) [noise=true , address=145.79.227.31:4808 , piece_id=6AICYVWNWGMAKTMLOOCBHIFLTVHL3YFVCV2LNEF65UTAX75347CQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: write tcp 188.166.225.170:52340->145.79.227.31:4808: use of closed network connection; write tcp 188.166.225.170:52340->145.79.227.31:4808: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52340->145.79.227.31:4808: use of closed network connection Sep 6 09:41:53.120433525 (187h33m49.800629084s) [noise=true , address=145.79.227.12:4792 , piece_id=GEFYSUVTBW32YENOIR5L5OEKRNY2HC4LEGBFDGQKWOQHNBB2SGDQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1fokaL7frEkFdVZRGmqaq3TmZVNYpt4agWCJZ8ABHfsYwkjAuv, address:145.79.227.12:4792): protocol: write tcp 188.166.225.170:33206->145.79.227.12:4792: use of closed network connection; write tcp 188.166.225.170:33206->145.79.227.12:4792: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33206->145.79.227.12:4792: use of closed network connection Sep 6 09:41:53.120468471 (187h33m49.80066404s) [noise=true , address=145.79.227.14:4870 , piece_id=MWBH2XLDE6Z6E7PGEVA53SB3ATETOOMFFOP5SHYLF22SX5JEQKRA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1MmdhnDp6kKktFb5btZUDeVQ2u3CNUKCZU7RfuEqiSQ2TEqwKX, address:145.79.227.14:4870): protocol: read tcp 188.166.225.170:43798->145.79.227.14:4870: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:53.120481857 (187h33m49.800677426s) [noise=true , address=145.79.227.45:4235 , piece_id=5ZPQS5SSHLPOZVDMRJWDS7QZ3ETUW5P77LIJTSKN6ZZQPCYIQF3A, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1a1sW1G1WPL5WQ2XxprWKQMsJkcb5ETSkk3ULp1TjRaZWptaT, address:145.79.227.45:4235): protocol: write tcp 188.166.225.170:38130->145.79.227.45:4235: use of closed network connection; write tcp 188.166.225.170:38130->145.79.227.45:4235: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38130->145.79.227.45:4235: use of closed network connection Sep 6 09:41:53.120500233 (187h33m49.800695802s) [noise=true , address=145.79.227.24:4610 , piece_id=DFCNN2DTCZMYR4N7NYWYSZGQOBFWTA7HDAR43VRAYKUSVG3RPMWA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1KwSqrCrryAnNkjhZkfZjcCetBLLE9EonzgWi3UkBYW4wefZ1B, address:145.79.227.24:4610): protocol: read tcp 188.166.225.170:55368->145.79.227.24:4610: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:53.120551199 (187h33m49.800746769s) [noise=true , address=145.79.227.13:4817 , piece_id=LDJJ5IVZBX3ABQQYXCZPNYLQVVIIMEBW2JPQ5I2ADLS3THKEHBPQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1RSBodzJxMye65QgkyrzkWCKQ49cnfdu4haUZKW23nWpqzMcYz, address:145.79.227.13:4817): protocol: write tcp 188.166.225.170:51768->145.79.227.13:4817: use of closed network connection; write tcp 188.166.225.170:51768->145.79.227.13:4817: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51768->145.79.227.13:4817: use of closed network connection Sep 6 09:41:53.120556240 (187h33m49.800751809s) [noise=true , address=145.79.227.44:4226 , piece_id=HHKETOY5XFFG26IPYXX7WCVLTEEY5QHBPLFSUA5P35CIREEFNXCQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1qN8jYqiZGuPjqHSp6siqdT4p33qvf1vYdAkMZKRnFgcQj9Ga3, address:145.79.227.44:4226): protocol: write tcp 188.166.225.170:33906->145.79.227.44:4226: use of closed network connection; write tcp 188.166.225.170:33906->145.79.227.44:4226: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33906->145.79.227.44:4226: use of closed network connection Sep 6 09:41:53.120623408 (187h33m49.800818977s) [noise=true , address=145.79.227.49:4353 , piece_id=2GFUHSUE2J5IHVKQHLEFRLRRRL3GOQ2XRHUFHUN4PJR5OVQQR7IA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:12BmzVLKbkqPMB7bJHGzFbgRRLu3TLg6TdTBCvYybsvz2yLpBbQ, address:145.79.227.49:4353): protocol: write tcp 188.166.225.170:43846->145.79.227.49:4353: use of closed network connection; write tcp 188.166.225.170:43846->145.79.227.49:4353: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43846->145.79.227.49:4353: use of closed network connection Sep 6 09:41:53.120678764 (187h33m49.800874333s) [noise=true , address=77.90.31.17:4803 , piece_id=Y7RZQJEF42WNLEK6GRLZ7FYUZOKYWVP3NGB55HRWF6RFYOUVUQTQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:12onacKUY32faZTFuyEjvMCZA4nVKiRKmTKh1WRFMfif2Kw1Q7Y, address:77.90.31.17:4803): protocol: write tcp 188.166.225.170:46258->77.90.31.17:4803: use of closed network connection; write tcp 188.166.225.170:46258->77.90.31.17:4803: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46258->77.90.31.17:4803: use of closed network connection Sep 6 09:41:53.120735432 (187h33m49.800931001s) [noise=true , address=45.91.116.6:4003 , piece_id=YPKJICQTFITRF3R4TWAJELZPBXFGUDSMHXZAEBCKRXCUU4NWA7VQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1veqoBGw6a8AGMoigyuNJMjcEvCT93xRq2B1d2KNhs6aEmSqVF, address:45.91.116.6:4003): protocol: read tcp 188.166.225.170:33250->45.91.116.6:4003: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:53.120745933 (187h33m49.800941502s) [noise=true , address=45.91.116.9:4283 , piece_id=ZROWQJZ5N2Z626VDED42TH3I2S2QKFZUSOBAMUJ6PC54TOXH3ZDA, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1bquxzprFQQwaeW4KbSR5i8CBP6XCcHAWPmAmoszcsLUWKKT5g, address:45.91.116.9:4283): protocol: write tcp 188.166.225.170:41256->45.91.116.9:4283: use of closed network connection; write tcp 188.166.225.170:41256->45.91.116.9:4283: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41256->45.91.116.9:4283: use of closed network connection Sep 6 09:41:53.120835033 (187h33m49.801030602s) [noise=true , address=145.79.227.19:4110 , piece_id=M7JEXISIEGLSZ6WJ6UKJMSROFWPYGNK6IMKHKWK7GW23GSAFOQCQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:1ADbnPJGRTw2NqRzhy7ptojGYoNdJgqmjuWBSjcrQK53Wm7LXi, address:145.79.227.19:4110): protocol: read tcp 188.166.225.170:46328->145.79.227.19:4110: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 09:41:53.120903534 (187h33m49.801099103s) [noise=true , address=145.79.227.41:4099 , piece_id=YZVL7SPX3AFNQOKCJDGLDOBDZB3EV2EDCLS6YBQS4JO7PZ5BQAAQ, seg_pos={0 0}, upload=165029]: Done uploading piece. err: ecclient: upload failed (node:122dGFWNmsuheRmfEfNGWHftPpCcWv7HHXKymgJxsye6P7ZKhZT, address:145.79.227.41:4099): protocol: read tcp 188.166.225.170:60952->145.79.227.41:4099: use of closed network connection; conn closed: op Sep 6 09:41:53.120931467 (187h33m49.801127036s) [seg_pos={0 0}, upload=165029]: Done waiting for segment. successful: 0 optimal: 14 errs: <nil> Duration: 1m0.448752051s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 3.747620977s FAIL (2025-09-06 09:41:53 UTC) orbiter warning: operation: Commit uplink: failed to upload enough pieces (needed at least 49 but got 46) Duration: 3.747649262s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 12 out of 60 is failed (threshold=3) [....................................X.X.X.X.XX.XXX.X..X...X.] duration: 22.157119834s FAIL (2025-09-06 09:41:39 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 22.157283878s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)