Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
686.693891ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:45:49 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 686.719881ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [X.X.........................................................] duration:
2.390269901s
FAIL (2025-08-26 06:45:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.390284479s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
242.275511ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:45:41 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 242.303296ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..XX.X......................................................] duration:
5.758915723s
FAIL (2025-08-26 06:45:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.758959207s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
714.190814ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:45:18 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 714.214801ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [X....X......................................................] duration:
239.328746ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:45:17 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 239.350778ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
3 out of 60 is failed (threshold=3) [XXX.........................................................] duration:
1.876080043s
FAIL (2025-08-26 06:45:12 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.876100762s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [XX.X........................................................] duration:
2.195251631s
FAIL (2025-08-26 06:45:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.195279444s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.XXX........................................................] duration:
1.878654389s
FAIL (2025-08-26 06:44:53 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.878673816s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.48156234s
FAIL (2025-08-26 06:44:52 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.481589923s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
1.856184495s
FAIL (2025-08-26 06:44:48 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.856211156s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [XXX.X.......................................................] duration:
2.191243962s
FAIL (2025-08-26 06:44:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.191265283s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [X..XX.X.....................................................] duration:
5.681926713s
FAIL (2025-08-26 06:44:34 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.681962331s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.668772623s
FAIL (2025-08-26 06:44:34 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 05:57:53.229787528 (13h54m20.388553262s) [upload=12181]: Starting upload
Aug 26 05:57:53.229866149 (13h54m20.388631883s) [upload=12181]: Done starting upload
Aug 26 05:57:53.229949559 (13h54m20.388715293s) [upload=12181]: Uploading segments...
Aug 26 05:57:53.230047747 (13h54m20.388813481s) [upload=12181]: Got next segment. Inline: false
Aug 26 05:57:53.230078435 (13h54m20.388844169s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:57:53.230083365 (13h54m20.388849099s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:57:57.671117492 (13h54m24.829883226s) [seg_pos={0 0}, upload=12181]: Begin upload segment...
Aug 26 05:57:57.671158381 (13h54m24.829924115s) [seg_pos={0 0}, upload=12181]: Done begin upload segment.
Aug 26 05:57:57.671178558 (13h54m24.829944293s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671225048 (13h54m24.829990782s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671552484 (13h54m24.830318208s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671741506 (13h54m24.83050724s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671838101 (13h54m24.830603835s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672083650 (13h54m24.830849384s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672177120 (13h54m24.830942854s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672491421 (13h54m24.831257155s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672508203 (13h54m24.831273937s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672788690 (13h54m24.831554414s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672887669 (13h54m24.831653403s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673083794 (13h54m24.831849518s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673200808 (13h54m24.831966542s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673305198 (13h54m24.832070922s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673460926 (13h54m24.83222666s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673715744 (13h54m24.832481468s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673839791 (13h54m24.832605525s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.674066565 (13h54m24.832832299s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:58.853667308 (13h54m26.012433042s) [upload=12181]: Next returned nil segment
Aug 26 05:57:58.853685563 (13h54m26.012451287s) [upload=12181]: Waiting for error group managing segments...
Aug 26 05:57:59.158597313 (13h54m26.317363047s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.530803022 (13h54m26.689568757s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.590970267 (13h54m26.749735991s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.591150923 (13h54m26.749916657s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.592999480 (13h54m26.751765214s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.609956145 (13h54m26.768721879s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.615646122 (13h54m26.774411846s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.618123673 (13h54m26.776889427s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.623700524 (13h54m26.782466258s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.626669584 (13h54m26.785435318s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.646945800 (13h54m26.805711535s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.651400024 (13h54m26.810165758s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.657389944 (13h54m26.816155678s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660101904 (13h54m26.818867638s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660121592 (13h54m26.818887326s) [seg_pos={0 0}, upload=12181]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:57:59.660309161 (13h54m26.819074895s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FmG5i2o4BwJmdL9u2tqKGmTAUErivGEe63TbfDTPpBdELozfz): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660359727 (13h54m26.819125462s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660436014 (13h54m26.819201748s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660500126 (13h54m26.81926586s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660521798 (13h54m26.819287532s) [seg_pos={0 0}, upload=12181]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 26 05:57:59.660580931 (13h54m26.819346665s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:57:59.660584878 (13h54m26.819350612s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:58:07.696182248 (13h54m34.854947982s) [upload=12181]: Done uploading segments...
Aug 26 05:58:07.696231492 (13h54m34.854997226s) [upload=12181]: Upload finished. err: <nil>
Duration: 5.668800036s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.XXX.X......................................................] duration:
2.249301919s
FAIL (2025-08-26 06:44:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.249325314s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
715.948226ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:44:17 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 715.977463ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
4 out of 60 is failed (threshold=3) [X.XXX.......................................................] duration:
1.892694383s
FAIL (2025-08-26 06:44:04 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.89271965s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.X....X.....................................................] duration:
238.908101ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:44:04 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 238.940292ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.483718956s
FAIL (2025-08-26 06:43:53 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.483745959s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
1.879612061s
FAIL (2025-08-26 06:43:53 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.879715209s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
4 out of 60 is failed (threshold=3) [..XXX.X.....................................................] duration:
2.310734908s
FAIL (2025-08-26 06:43:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.310756178s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
5.483056822s
FAIL (2025-08-26 06:43:27 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 05:57:53.229787528 (13h54m20.388553262s) [upload=12181]: Starting upload
Aug 26 05:57:53.229866149 (13h54m20.388631883s) [upload=12181]: Done starting upload
Aug 26 05:57:53.229949559 (13h54m20.388715293s) [upload=12181]: Uploading segments...
Aug 26 05:57:53.230047747 (13h54m20.388813481s) [upload=12181]: Got next segment. Inline: false
Aug 26 05:57:53.230078435 (13h54m20.388844169s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:57:53.230083365 (13h54m20.388849099s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:57:57.671117492 (13h54m24.829883226s) [seg_pos={0 0}, upload=12181]: Begin upload segment...
Aug 26 05:57:57.671158381 (13h54m24.829924115s) [seg_pos={0 0}, upload=12181]: Done begin upload segment.
Aug 26 05:57:57.671178558 (13h54m24.829944293s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671225048 (13h54m24.829990782s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671552484 (13h54m24.830318208s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671741506 (13h54m24.83050724s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671838101 (13h54m24.830603835s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672083650 (13h54m24.830849384s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672177120 (13h54m24.830942854s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672491421 (13h54m24.831257155s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672508203 (13h54m24.831273937s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672788690 (13h54m24.831554414s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672887669 (13h54m24.831653403s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673083794 (13h54m24.831849518s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673200808 (13h54m24.831966542s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673305198 (13h54m24.832070922s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673460926 (13h54m24.83222666s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673715744 (13h54m24.832481468s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673839791 (13h54m24.832605525s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.674066565 (13h54m24.832832299s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:58.853667308 (13h54m26.012433042s) [upload=12181]: Next returned nil segment
Aug 26 05:57:58.853685563 (13h54m26.012451287s) [upload=12181]: Waiting for error group managing segments...
Aug 26 05:57:59.158597313 (13h54m26.317363047s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.530803022 (13h54m26.689568757s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.590970267 (13h54m26.749735991s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.591150923 (13h54m26.749916657s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.592999480 (13h54m26.751765214s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.609956145 (13h54m26.768721879s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.615646122 (13h54m26.774411846s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.618123673 (13h54m26.776889427s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.623700524 (13h54m26.782466258s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.626669584 (13h54m26.785435318s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.646945800 (13h54m26.805711535s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.651400024 (13h54m26.810165758s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.657389944 (13h54m26.816155678s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660101904 (13h54m26.818867638s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660121592 (13h54m26.818887326s) [seg_pos={0 0}, upload=12181]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:57:59.660309161 (13h54m26.819074895s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FmG5i2o4BwJmdL9u2tqKGmTAUErivGEe63TbfDTPpBdELozfz): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660359727 (13h54m26.819125462s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660436014 (13h54m26.819201748s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660500126 (13h54m26.81926586s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660521798 (13h54m26.819287532s) [seg_pos={0 0}, upload=12181]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 26 05:57:59.660580931 (13h54m26.819346665s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:57:59.660584878 (13h54m26.819350612s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:58:07.696182248 (13h54m34.854947982s) [upload=12181]: Done uploading segments...
Aug 26 05:58:07.696231492 (13h54m34.854997226s) [upload=12181]: Upload finished. err: <nil>
Duration: 5.483086388s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [X..X........................................................] duration:
710.792995ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:43:27 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 710.817491ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
4 out of 60 is failed (threshold=3) [.X.XXX......................................................] duration:
1.875178098s
FAIL (2025-08-26 06:43:22 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.8752004s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [XX..XX.X....................................................] duration:
5.748086716s
FAIL (2025-08-26 06:43:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.74811548s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..X....X....................................................] duration:
235.582428ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:43:18 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 235.616974ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
2.067806224s
FAIL (2025-08-26 06:43:16 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 2.067833377s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................] duration:
5.720465081s
FAIL (2025-08-26 06:43:07 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 05:57:53.229787528 (13h54m20.388553262s) [upload=12181]: Starting upload
Aug 26 05:57:53.229866149 (13h54m20.388631883s) [upload=12181]: Done starting upload
Aug 26 05:57:53.229949559 (13h54m20.388715293s) [upload=12181]: Uploading segments...
Aug 26 05:57:53.230047747 (13h54m20.388813481s) [upload=12181]: Got next segment. Inline: false
Aug 26 05:57:53.230078435 (13h54m20.388844169s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:57:53.230083365 (13h54m20.388849099s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:57:57.671117492 (13h54m24.829883226s) [seg_pos={0 0}, upload=12181]: Begin upload segment...
Aug 26 05:57:57.671158381 (13h54m24.829924115s) [seg_pos={0 0}, upload=12181]: Done begin upload segment.
Aug 26 05:57:57.671178558 (13h54m24.829944293s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671225048 (13h54m24.829990782s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671552484 (13h54m24.830318208s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671741506 (13h54m24.83050724s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671838101 (13h54m24.830603835s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672083650 (13h54m24.830849384s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672177120 (13h54m24.830942854s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672491421 (13h54m24.831257155s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672508203 (13h54m24.831273937s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672788690 (13h54m24.831554414s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672887669 (13h54m24.831653403s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673083794 (13h54m24.831849518s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673200808 (13h54m24.831966542s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673305198 (13h54m24.832070922s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673460926 (13h54m24.83222666s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673715744 (13h54m24.832481468s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673839791 (13h54m24.832605525s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.674066565 (13h54m24.832832299s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:58.853667308 (13h54m26.012433042s) [upload=12181]: Next returned nil segment
Aug 26 05:57:58.853685563 (13h54m26.012451287s) [upload=12181]: Waiting for error group managing segments...
Aug 26 05:57:59.158597313 (13h54m26.317363047s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.530803022 (13h54m26.689568757s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.590970267 (13h54m26.749735991s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.591150923 (13h54m26.749916657s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.592999480 (13h54m26.751765214s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.609956145 (13h54m26.768721879s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.615646122 (13h54m26.774411846s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.618123673 (13h54m26.776889427s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.623700524 (13h54m26.782466258s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.626669584 (13h54m26.785435318s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.646945800 (13h54m26.805711535s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.651400024 (13h54m26.810165758s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.657389944 (13h54m26.816155678s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660101904 (13h54m26.818867638s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660121592 (13h54m26.818887326s) [seg_pos={0 0}, upload=12181]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:57:59.660309161 (13h54m26.819074895s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FmG5i2o4BwJmdL9u2tqKGmTAUErivGEe63TbfDTPpBdELozfz): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660359727 (13h54m26.819125462s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660436014 (13h54m26.819201748s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660500126 (13h54m26.81926586s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660521798 (13h54m26.819287532s) [seg_pos={0 0}, upload=12181]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 26 05:57:59.660580931 (13h54m26.819346665s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:57:59.660584878 (13h54m26.819350612s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:58:07.696182248 (13h54m34.854947982s) [upload=12181]: Done uploading segments...
Aug 26 05:58:07.696231492 (13h54m34.854997226s) [upload=12181]: Upload finished. err: <nil>
Duration: 5.720494399s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
5 out of 60 is failed (threshold=3) [X.X.XXX.....................................................] duration:
1.857173257s
FAIL (2025-08-26 06:43:01 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.857204136s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.XX..XX.X...................................................] duration:
5.635716888s
FAIL (2025-08-26 06:42:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.635744851s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.X.........................................................] duration:
5.495725873s
FAIL (2025-08-26 06:42:41 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.495753076s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [X.....XX....................................................] duration:
1.828265164s
FAIL (2025-08-26 06:42:38 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.828292095s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration:
719.24231ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:42:37 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 719.261497ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
5 out of 60 is failed (threshold=3) [X..XXX.X....................................................] duration:
2.22867881s
FAIL (2025-08-26 06:42:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.228705793s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [..XX..XX.X..................................................] duration:
5.756042444s
FAIL (2025-08-26 06:42:25 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.756074877s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
6 out of 60 is failed (threshold=3) [XX.X.XXX....................................................] duration:
1.858938082s
FAIL (2025-08-26 06:42:23 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.858959853s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...X....X...................................................] duration:
231.054683ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:42:22 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 231.083078ms
edge
linksharing
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
6.27059059s
FAIL (2025-08-26 06:42:09 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.270610599s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
3 out of 60 is failed (threshold=3) [XX.X........................................................] duration:
5.641499969s
FAIL (2025-08-26 06:42:06 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.64152641s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
5.571884605s
FAIL (2025-08-26 06:41:58 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 05:57:53.229787528 (13h54m20.388553262s) [upload=12181]: Starting upload
Aug 26 05:57:53.229866149 (13h54m20.388631883s) [upload=12181]: Done starting upload
Aug 26 05:57:53.229949559 (13h54m20.388715293s) [upload=12181]: Uploading segments...
Aug 26 05:57:53.230047747 (13h54m20.388813481s) [upload=12181]: Got next segment. Inline: false
Aug 26 05:57:53.230078435 (13h54m20.388844169s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:57:53.230083365 (13h54m20.388849099s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:57:57.671117492 (13h54m24.829883226s) [seg_pos={0 0}, upload=12181]: Begin upload segment...
Aug 26 05:57:57.671158381 (13h54m24.829924115s) [seg_pos={0 0}, upload=12181]: Done begin upload segment.
Aug 26 05:57:57.671178558 (13h54m24.829944293s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671225048 (13h54m24.829990782s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671552484 (13h54m24.830318208s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671741506 (13h54m24.83050724s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671838101 (13h54m24.830603835s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672083650 (13h54m24.830849384s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672177120 (13h54m24.830942854s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672491421 (13h54m24.831257155s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672508203 (13h54m24.831273937s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672788690 (13h54m24.831554414s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672887669 (13h54m24.831653403s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673083794 (13h54m24.831849518s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673200808 (13h54m24.831966542s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673305198 (13h54m24.832070922s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673460926 (13h54m24.83222666s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673715744 (13h54m24.832481468s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673839791 (13h54m24.832605525s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.674066565 (13h54m24.832832299s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:58.853667308 (13h54m26.012433042s) [upload=12181]: Next returned nil segment
Aug 26 05:57:58.853685563 (13h54m26.012451287s) [upload=12181]: Waiting for error group managing segments...
Aug 26 05:57:59.158597313 (13h54m26.317363047s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.530803022 (13h54m26.689568757s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.590970267 (13h54m26.749735991s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.591150923 (13h54m26.749916657s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.592999480 (13h54m26.751765214s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.609956145 (13h54m26.768721879s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.615646122 (13h54m26.774411846s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.618123673 (13h54m26.776889427s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.623700524 (13h54m26.782466258s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.626669584 (13h54m26.785435318s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.646945800 (13h54m26.805711535s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.651400024 (13h54m26.810165758s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.657389944 (13h54m26.816155678s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660101904 (13h54m26.818867638s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660121592 (13h54m26.818887326s) [seg_pos={0 0}, upload=12181]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:57:59.660309161 (13h54m26.819074895s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FmG5i2o4BwJmdL9u2tqKGmTAUErivGEe63TbfDTPpBdELozfz): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660359727 (13h54m26.819125462s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660436014 (13h54m26.819201748s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660500126 (13h54m26.81926586s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660521798 (13h54m26.819287532s) [seg_pos={0 0}, upload=12181]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 26 05:57:59.660580931 (13h54m26.819346665s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:57:59.660584878 (13h54m26.819350612s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:58:07.696182248 (13h54m34.854947982s) [upload=12181]: Done uploading segments...
Aug 26 05:58:07.696231492 (13h54m34.854997226s) [upload=12181]: Upload finished. err: <nil>
Duration: 5.571912057s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X.....XX...................................................] duration:
1.837746412s
FAIL (2025-08-26 06:41:48 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.837876932s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [..X..X......................................................] duration:
700.715581ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=433 Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 06:00:36 GMT Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=185F3BB6D312505B X-Request-Id=6Gw8NdfoL57 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:41:48 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3BB6D312505B</RequestId><HostId></HostId></Error>"
Duration: 700.737583ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.XX.X.......................................................] duration:
5.544129409s
FAIL (2025-08-26 06:41:39 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.544156099s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [....X....X..................................................] duration:
247.072131ms remote:
"tcp:185.244.226.2:443" response headers:
Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Date=Tue, 26 Aug 2025 06:00:37 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Origin=* X-Request-Id=77T4Bv5mHju target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:41:34 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error | Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error | Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error | Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 247.09777ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [X..XX..XX.X.................................................] duration:
5.86289992s
FAIL (2025-08-26 06:41:28 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:33.966981454 (13h58m1.125747188s) [upload=12238]: Starting upload
Aug 26 06:01:33.967043734 (13h58m1.125809468s) [upload=12238]: Done starting upload
Aug 26 06:01:33.967134407 (13h58m1.125900141s) [upload=12238]: Uploading segments...
Aug 26 06:01:33.967231903 (13h58m1.125997637s) [upload=12238]: Got next segment. Inline: false
Aug 26 06:01:33.967265127 (13h58m1.126030851s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:33.967269485 (13h58m1.126035219s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:36.228682523 (13h58m3.387448257s) [seg_pos={0 0}, upload=12238]: Begin upload segment...
Aug 26 06:01:36.228721969 (13h58m3.387487693s) [seg_pos={0 0}, upload=12238]: Done begin upload segment.
Aug 26 06:01:36.228741846 (13h58m3.38750757s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228749832 (13h58m3.387515566s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.228790529 (13h58m3.387556264s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229133707 (13h58m3.387899441s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229175256 (13h58m3.38794099s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229559611 (13h58m3.388325355s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229859685 (13h58m3.388625419s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.229933657 (13h58m3.388699381s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230227839 (13h58m3.388993563s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230234873 (13h58m3.389000607s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230512725 (13h58m3.389278449s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230645098 (13h58m3.389410832s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.230829510 (13h58m3.389595245s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231136609 (13h58m3.389902343s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231242481 (13h58m3.390008225s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:36.231384423 (13h58m3.390150147s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Uploading piece...
Aug 26 06:01:37.971564373 (13h58m5.130330107s) [upload=12238]: Next returned nil segment
Aug 26 06:01:37.971583579 (13h58m5.130349313s) [upload=12238]: Waiting for error group managing segments...
Aug 26 06:01:38.251877415 (13h58m5.410643139s) [noise=true , address=45.91.116.14:4190 , piece_id=4Z4ENOGXEXOBEGS23D3CNMDLD7KKTPRCF4UUEEYJIDURLB5NPV2Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.252547697 (13h58m5.411313431s) [noise=true , address=45.91.116.12:4226 , piece_id=A6YHKD5AFXRKOWQWGFKOMDLPU3OMGKSV2TVM4NRJC35GWZPD7JAQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.261324140 (13h58m5.420089874s) [noise=true , address=45.91.116.13:4155 , piece_id=ZQPF33BDGT3H6WQSI3P7VCFPGE5SF2VXJGPKBXBKENL3MSRRJM7A, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.262792210 (13h58m5.421557944s) [noise=true , address=77.90.31.19:4841 , piece_id=ABB6E26HC74OUMTDONDWVIR3J7KQGDRJPWLE3WUPSE5OCQZXSD3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.267644835 (13h58m5.426410569s) [noise=true , address=45.91.116.13:4153 , piece_id=5KVUFP43OT4TBGY6ER7AYPA7GJHLBXURLRBCTJNH2VJM6H27XLCQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.269180032 (13h58m5.427945766s) [noise=true , address=45.91.116.7:4009 , piece_id=YWY62TMLMAC4RO3VL32S4Y4AICDG23IMLGJFLN5O5AXWJHZOREQA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.277360584 (13h58m5.436126318s) [noise=true , address=45.91.116.9:4261 , piece_id=O6EYBDQ346LA2BRQ2JXV3EZY6MIKRGTMY2LMSG7W55PCQ5NOYFHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.288401611 (13h58m5.447167335s) [noise=true , address=45.91.116.7:4011 , piece_id=N2JHFX2J2N4HJQMSGDW63KTSYRUNZ5XSSEGPAXDNC7APTVDDXZEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.326794461 (13h58m5.485560195s) [noise=true , address=45.91.116.10:4083 , piece_id=BKKDIJSBDXPFIEBFAE5K3HMZYMUV57DZZZ2PPTKSDN5W26HRUG3Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.327600795 (13h58m5.486366529s) [noise=true , address=77.90.31.12:4590 , piece_id=TKUT2ZV7CIAF4WODX4BS3PVMUFPUB3S4Q5VWLWSOXMTQH3WSCEVA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.330252889 (13h58m5.489018633s) [noise=true , address=77.90.31.14:4663 , piece_id=KTCI2YDX527DYFGHLU3EN7X5CET52EPI3OPZMMM23GVUC6UEWPEA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.337759492 (13h58m5.496525226s) [noise=true , address=104.36.82.10:4077 , piece_id=SWDDXFHVEAASFT3QGRVXNJREDIIKXGJWKT2AMC5WBRV3QI356SGQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341411059 (13h58m5.500176803s) [noise=true , address=45.91.116.4:4940 , piece_id=EOTFWQ7IA3ZGOKGV6UL76IW3KVBH7BSDUWNJJANSH3RIXV4P3XHQ, seg_pos={0 0}, upload=12238]: Done uploading piece. err: <nil>
Aug 26 06:01:38.341436287 (13h58m5.500202021s) [seg_pos={0 0}, upload=12238]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:38.341547309 (13h58m5.500313043s) [noise=true , address=77.90.31.21:4878 , piece_id=HGMW355XDVQ7C5V3S4TJA4TIETBKSN7TPS4P4GUBQICLN4QAEWPA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341666147 (13h58m5.500431882s) [noise=true , address=77.90.31.16:4734 , piece_id=UJMKMSAHDH5X74RPMOKRR2SHTWZNMJPVBCOFV3T3OTCP2DOJR7JA, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341725080 (13h58m5.500490814s) [noise=true , address=77.90.31.18:4805 , piece_id=AETDASIPJ37ICCK4EKSLOVLS77S22JBI4GVSBASTZPPSSFZDXK4Q, seg_pos={0 0}, upload=12238]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:38.341751140 (13h58m5.500516874s) [seg_pos={0 0}, upload=12238]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:38.341841753 (13h58m5.500607487s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:38.341846252 (13h58m5.500611986s) [upload=12238]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:43.222504557 (13h58m10.381270291s) [upload=12238]: Done uploading segments...
Aug 26 06:01:43.222522181 (13h58m10.381287915s) [upload=12238]: Upload finished. err: <nil>
Duration: 5.862925651s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
3 out of 60 is failed (threshold=3) [..X.....XX..................................................] duration:
1.852963022s
FAIL (2025-08-26 06:41:28 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.852999172s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
6 out of 60 is failed (threshold=3) [.XX.X.XXX...................................................] duration:
1.875573594s
FAIL (2025-08-26 06:41:26 UTC)
orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:08.342560001 (13h57m35.501325736s) [upload=12234]: Starting upload Aug 26 06:01:08.342643181 (13h57m35.501408915s) [upload=12234]: Done starting upload Aug 26 06:01:08.342701543 (13h57m35.501467277s) [upload=12234]: Uploading segments... Aug 26 06:01:08.342775774 (13h57m35.501541509s) [upload=12234]: Got next segment. Inline: true Aug 26 06:01:08.342780554 (13h57m35.501546289s) [upload=12234]: Waiting for error group managing segments... Aug 26 06:01:08.342822424 (13h57m35.501588158s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 26 06:01:08.342827043 (13h57m35.501592777s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 26 06:01:08.342829718 (13h57m35.501595452s) [upload=12234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 26 06:01:12.739727752 (13h57m39.898493507s) [upload=12234]: Done uploading segments... Aug 26 06:01:12.739748322 (13h57m39.898514056s) [upload=12234]: Upload finished. err: <nil>
Duration: 1.875607859s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
5 out of 60 is failed (threshold=3) [.X..XXX.X...................................................] duration:
2.175062853s
FAIL (2025-08-26 06:41:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.175082771s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................] duration:
5.808624472s
FAIL (2025-08-26 06:41:16 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.80864928s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
5.765653248s
FAIL (2025-08-26 06:41:03 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 05:57:53.229787528 (13h54m20.388553262s) [upload=12181]: Starting upload
Aug 26 05:57:53.229866149 (13h54m20.388631883s) [upload=12181]: Done starting upload
Aug 26 05:57:53.229949559 (13h54m20.388715293s) [upload=12181]: Uploading segments...
Aug 26 05:57:53.230047747 (13h54m20.388813481s) [upload=12181]: Got next segment. Inline: false
Aug 26 05:57:53.230078435 (13h54m20.388844169s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:57:53.230083365 (13h54m20.388849099s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:57:57.671117492 (13h54m24.829883226s) [seg_pos={0 0}, upload=12181]: Begin upload segment...
Aug 26 05:57:57.671158381 (13h54m24.829924115s) [seg_pos={0 0}, upload=12181]: Done begin upload segment.
Aug 26 05:57:57.671178558 (13h54m24.829944293s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671225048 (13h54m24.829990782s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671552484 (13h54m24.830318208s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671741506 (13h54m24.83050724s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.671838101 (13h54m24.830603835s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672083650 (13h54m24.830849384s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672177120 (13h54m24.830942854s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672491421 (13h54m24.831257155s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672508203 (13h54m24.831273937s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672788690 (13h54m24.831554414s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.672887669 (13h54m24.831653403s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673083794 (13h54m24.831849518s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673200808 (13h54m24.831966542s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673305198 (13h54m24.832070922s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673460926 (13h54m24.83222666s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673715744 (13h54m24.832481468s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.673839791 (13h54m24.832605525s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:57.674066565 (13h54m24.832832299s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Uploading piece...
Aug 26 05:57:58.853667308 (13h54m26.012433042s) [upload=12181]: Next returned nil segment
Aug 26 05:57:58.853685563 (13h54m26.012451287s) [upload=12181]: Waiting for error group managing segments...
Aug 26 05:57:59.158597313 (13h54m26.317363047s) [noise=true , address=64.49.13.154:4934 , piece_id=T55ATJHP6AO2BIKFYO6WBOLERUI7FNFUCTSOQ4ASPD6FNDJKJWSA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.530803022 (13h54m26.689568757s) [noise=true , address=79.127.219.51:4199 , piece_id=HIRZ2XKQBZ64DP3C37LGLKOZOZKI4EF6MGQYMNIQGKPZSJCWXTYQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.590970267 (13h54m26.749735991s) [noise=true , address=45.91.116.4:4960 , piece_id=7X22MD7TCFLRW4G47Q4P3HRSF2QNQCGOYWJBUFTSM66OSFBRQDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.591150923 (13h54m26.749916657s) [noise=true , address=104.36.82.50:4028 , piece_id=ZF55MSIK6YVWAMZZVADOCOEMGWGV3KVF57DMBUIC5FW4G5H576ZA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.592999480 (13h54m26.751765214s) [noise=true , address=104.36.82.60:4363 , piece_id=J7NKL4MGXJGQPYRSOHJII3ONXFXZLYON5SAH5KYIZKG7XM3RBGAQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.609956145 (13h54m26.768721879s) [noise=true , address=156.146.43.225:4450 , piece_id=P3CORXRBNMVYZR3M2VJWPDE5LKL6BYAK2VOW6PW6FO7GZNXPHHHQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.615646122 (13h54m26.774411846s) [noise=true , address=104.36.82.10:4088 , piece_id=CPDOJATUHTB5OFXSYTI2JFQ5ZSXSJCOV5B6DU3HILFJNIHGNYMRA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.618123673 (13h54m26.776889427s) [noise=true , address=104.36.82.5:4857 , piece_id=UYH3F2UZXZPJS33QLL2TFF3QMT3RSVGIBZDPHJMTGPH5VLAZ27OA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.623700524 (13h54m26.782466258s) [noise=true , address=77.90.31.14:4681 , piece_id=MPGCU5ZSZEL4VKCOYMS7ZX5A57BY46FLU4P3CQ4KITEWWOUDJ4LQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.626669584 (13h54m26.785435318s) [noise=true , address=104.36.82.44:4880 , piece_id=CCPOVQB7AUXKAXMVHLITUB6GVCFNKBCORWXOC7W5A54CDR2OFASA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.646945800 (13h54m26.805711535s) [noise=true , address=104.36.82.23:4468 , piece_id=OD45WJDG5OKB3IC4BULXPYHD6V7QTJMLO64KFWPGA6ZR7PO5ZBOQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.651400024 (13h54m26.810165758s) [noise=true , address=104.36.82.16:4315 , piece_id=E2KJWAQ5YZLL3ZEGBEV4U2TL5UVDGM5Y7AF2WTOYPNKI7I7WACVA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.657389944 (13h54m26.816155678s) [noise=true , address=77.90.31.18:4823 , piece_id=SCDSAFCYWBDBAA7PMCW2OTLPMXX3ZTVQEAMGQDDKMZLVQ3JFELKA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660101904 (13h54m26.818867638s) [noise=true , address=45.91.116.12:4236 , piece_id=VYYTQ2L4NJPJYBOCKOT7MW4Q2IPPQY3GUMUYV56NQPRBQRQEP3YA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660121592 (13h54m26.818887326s) [seg_pos={0 0}, upload=12181]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:57:59.660309161 (13h54m26.819074895s) [noise=true , address=104.36.82.15:4290 , piece_id=WN3XAYHTTP7MLRD4JBWZA46LCU7ZZOCJG33VXTM4MKVQAMDT4D3A, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FmG5i2o4BwJmdL9u2tqKGmTAUErivGEe63TbfDTPpBdELozfz): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660359727 (13h54m26.819125462s) [noise=true , address=77.90.31.19:4862 , piece_id=FEQAYMXTAZTNP3PCTIFJMW7IOWAZ6NRBMLARTS3IUCR2CHKYMKLA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: <nil>
Aug 26 05:57:59.660436014 (13h54m26.819201748s) [noise=true , address=79.127.201.215:4360 , piece_id=NPFLNT5ZXPXHL2G4SBO7G6PBBCED4DVTKUQ7H4AACEBIMHM2E4VQ, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660500126 (13h54m26.81926586s) [noise=true , address=77.90.31.10:4538 , piece_id=3AQWXQXBHCNYP2XQKCA4LPEDH3ZZ6WHVAYK3LTZS4WNP76DTWDEA, seg_pos={0 0}, upload=12181]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kRwLj3eRSu1a8oRSnUF5NuG6sJ9R9qMmFT89CHXMWB11WQD1N): protocol: expected piece hash; context canceled; EOF
Aug 26 05:57:59.660521798 (13h54m26.819287532s) [seg_pos={0 0}, upload=12181]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 26 05:57:59.660580931 (13h54m26.819346665s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:57:59.660584878 (13h54m26.819350612s) [upload=12181]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:58:07.696182248 (13h54m34.854947982s) [upload=12181]: Done uploading segments...
Aug 26 05:58:07.696231492 (13h54m34.854997226s) [upload=12181]: Upload finished. err: <nil>
Duration: 5.765681672s
core
uplink
us-select-1
us1-placement-20
|