Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [.....................XX...X......X..........X........XX.XX..] duration: 4.722185009s FAIL (2025-08-26 05:41:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:40:22.385961260 (13h36m49.544726994s) [upload=11919]: Starting upload Aug 26 05:40:22.386018800 (13h36m49.544784534s) [upload=11919]: Done starting upload Aug 26 05:40:22.386072673 (13h36m49.544838407s) [upload=11919]: Uploading segments... Aug 26 05:40:22.386111086 (13h36m49.54487681s) [upload=11919]: Got next segment. Inline: true Aug 26 05:40:22.386115005 (13h36m49.544880739s) [upload=11919]: Waiting for error group managing segments... Aug 26 05:40:22.386155261 (13h36m49.544920995s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:22.386161824 (13h36m49.544927558s) [upload=11919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:22.386164499 (13h36m49.544930233s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:23.933385976 (13h36m51.09215171s) [upload=11919]: Done uploading segments... Aug 26 05:40:23.933404712 (13h36m51.092170446s) [upload=11919]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 4.722210468s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 6 out of 60 is failed (threshold=6) [....X................XX...........X...X.................X...] duration: 1.919137169s remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* X-Request-Id=DS54VnSBYuP Date=Tue, 26 Aug 2025 05:38:26 GMT Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:41:24 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 1.919160935s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [..............X.X...XXX....XXX.......XXX.X..X....X....X.XXXX] duration: 30.001093487s FAIL (2025-08-26 05:41:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:40:54.413989195 (13h37m21.572754929s) [upload=11929]: Starting upload Aug 26 05:40:54.414037668 (13h37m21.572803392s) [upload=11929]: Done starting upload Aug 26 05:40:54.414586297 (13h37m21.573352032s) [upload=11929]: Uploading segments... Aug 26 05:40:54.415177699 (13h37m21.573943434s) [upload=11929]: Got next segment. Inline: false Aug 26 05:40:54.415234749 (13h37m21.574000483s) [upload=11929]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:54.415240008 (13h37m21.574005742s) [upload=11929]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:40:58.652520649 (13h37m25.811286383s) [seg_pos={0 0}, upload=11929]: Begin upload segment... Aug 26 05:40:58.652575635 (13h37m25.811341369s) [seg_pos={0 0}, upload=11929]: Done begin upload segment. Aug 26 05:40:58.652640980 (13h37m25.811406704s) [noise=true , address=77.90.31.12:4589 , piece_id=NTXF7QBJNFHITIXIORSWPVIETZ34BTOPM6LE26WW6BP3Y4N66OBQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.652658643 (13h37m25.811424367s) [noise=true , address=104.36.82.9:4006 , piece_id=DXAOC2MPR5QWV3VSFEWSZ5QAYEHE4U4DZZTIWRBWJVXRVUNWGHJQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.653137139 (13h37m25.811902874s) [noise=true , address=77.90.31.14:4663 , piece_id=CD6AMYO5DP3Z3UQCSFHSGTAUJR67L7N7VHVEHPPVCMSZTSEMVSPA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.653197986 (13h37m25.81196372s) [noise=true , address=45.91.116.4:4939 , piece_id=OGBRZWMJSVEZO3JCDV5PAZMC2IYSGSUFSLL3KN37JZ7GO73EFVPA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.653525292 (13h37m25.812291026s) [noise=true , address=77.90.31.15:4698 , piece_id=CYPJS4S2SDXFN6RZQEPNKZPISOQCCB32IQQVJSMAX5EZGDDIJI7A, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.653607870 (13h37m25.812373604s) [noise=true , address=77.90.31.18:4806 , piece_id=232IXWRQLX7QQF5FJZ7AOTUNIGALHJHUI4LYDP7BRT6GFGHKXOTQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.653900310 (13h37m25.812666034s) [noise=true , address=77.90.31.14:4661 , piece_id=RNZEZT36M7YEK2GPLAW7QR265YFQMSLTSQUDD7GMG5JZ7NQVJMXA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654008577 (13h37m25.812774311s) [noise=true , address=45.91.116.14:4191 , piece_id=UM7YHURTCIFKO62EEC556NQDQUPOU2SXWI65WRUBLFMU77CDCLNQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654198661 (13h37m25.812964385s) [noise=true , address=45.91.116.14:4190 , piece_id=VYU5YFWFWLMJ27FPXIN3O7SYFGD7PR5PSGRUDG3VJHEH2WLEPZWA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654260769 (13h37m25.813026493s) [noise=true , address=77.90.31.16:4734 , piece_id=CWHGVX6YBZTR4YESWWTZP567SFL6EXWI7XYJFLN7IPJPAXUO4WIQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654475130 (13h37m25.813240854s) [noise=true , address=45.91.116.9:4263 , piece_id=ZGWKEOEWLAADDOPVEKTXQEZQKJOKCO5LZ4PPK6FYF64AYG6DBIBA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654485219 (13h37m25.813250953s) [noise=true , address=45.91.116.5:4297 , piece_id=BYVGT4YLIB7W6CEAJMS3T3HKTJJJFTBW6UHFK2PSK62UTYQA34XA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654746169 (13h37m25.813511903s) [noise=true , address=77.90.31.21:4877 , piece_id=UDJNHTBDH2KPPAU2OJIB463LK5IG3SFT2X5PMQD54LDYOXLM56QA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.654780003 (13h37m25.813545727s) [noise=true , address=45.91.116.4:4937 , piece_id=ZFHKKRSLDBTGTKB2PTIBJCCOZC5S2OJJSNXMF7GVFMVAWLULLESA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.655046422 (13h37m25.813812156s) [noise=true , address=45.91.116.11:4118 , piece_id=4V75QEXPH3SYMOJ5TFOV6OBGYSL3ONNYZBMTA47UKEVZBQJRUCIQ, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:40:58.655264760 (13h37m25.814030494s) [noise=true , address=104.36.82.6:4874 , piece_id=NWJB3U6NL2TYDR5PWJE2NNVFCLZIBADLXZBEBT6RUEVLRPWD5VRA, seg_pos={0 0}, upload=11929]: Uploading piece... Aug 26 05:41:00.385208368 (13h37m27.543974102s) [upload=11929]: Next returned nil segment Aug 26 05:41:00.385225471 (13h37m27.543991205s) [upload=11929]: Waiting for error group managing segments... Aug 26 05:41:00.679914119 (13h37m27.838679853s) [noise=true , address=45.91.116.14:4191 , piece_id=UM7YHURTCIFKO62EEC556NQDQUPOU2SXWI65WRUBLFMU77CDCLNQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.680363789 (13h37m27.839129523s) [noise=true , address=45.91.116.14:4190 , piece_id=VYU5YFWFWLMJ27FPXIN3O7SYFGD7PR5PSGRUDG3VJHEH2WLEPZWA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.684301955 (13h37m27.843067689s) [noise=true , address=45.91.116.4:4937 , piece_id=ZFHKKRSLDBTGTKB2PTIBJCCOZC5S2OJJSNXMF7GVFMVAWLULLESA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.692412053 (13h37m27.851177787s) [noise=true , address=45.91.116.11:4118 , piece_id=4V75QEXPH3SYMOJ5TFOV6OBGYSL3ONNYZBMTA47UKEVZBQJRUCIQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.730785847 (13h37m27.889551581s) [noise=true , address=77.90.31.15:4698 , piece_id=CYPJS4S2SDXFN6RZQEPNKZPISOQCCB32IQQVJSMAX5EZGDDIJI7A, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.734784328 (13h37m27.893550062s) [noise=true , address=45.91.116.4:4939 , piece_id=OGBRZWMJSVEZO3JCDV5PAZMC2IYSGSUFSLL3KN37JZ7GO73EFVPA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.739532994 (13h37m27.898298728s) [noise=true , address=77.90.31.21:4877 , piece_id=UDJNHTBDH2KPPAU2OJIB463LK5IG3SFT2X5PMQD54LDYOXLM56QA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.742053547 (13h37m27.900819281s) [noise=true , address=77.90.31.14:4663 , piece_id=CD6AMYO5DP3Z3UQCSFHSGTAUJR67L7N7VHVEHPPVCMSZTSEMVSPA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.748805796 (13h37m27.90757153s) [noise=true , address=77.90.31.18:4806 , piece_id=232IXWRQLX7QQF5FJZ7AOTUNIGALHJHUI4LYDP7BRT6GFGHKXOTQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.770677135 (13h37m27.929442869s) [noise=true , address=45.91.116.5:4297 , piece_id=BYVGT4YLIB7W6CEAJMS3T3HKTJJJFTBW6UHFK2PSK62UTYQA34XA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.774735510 (13h37m27.933501245s) [noise=true , address=104.36.82.6:4874 , piece_id=NWJB3U6NL2TYDR5PWJE2NNVFCLZIBADLXZBEBT6RUEVLRPWD5VRA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.785123567 (13h37m27.943889301s) [noise=true , address=77.90.31.12:4589 , piece_id=NTXF7QBJNFHITIXIORSWPVIETZ34BTOPM6LE26WW6BP3Y4N66OBQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.791920281 (13h37m27.950686015s) [noise=true , address=77.90.31.16:4734 , piece_id=CWHGVX6YBZTR4YESWWTZP567SFL6EXWI7XYJFLN7IPJPAXUO4WIQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: <nil> Aug 26 05:41:00.791956952 (13h37m27.950722696s) [seg_pos={0 0}, upload=11929]: Segment reached optimal threshold of 13 pieces. Aug 26 05:41:00.792111988 (13h37m27.950877722s) [noise=true , address=77.90.31.14:4661 , piece_id=RNZEZT36M7YEK2GPLAW7QR265YFQMSLTSQUDD7GMG5JZ7NQVJMXA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Aug 26 05:41:00.792265222 (13h37m27.951030956s) [noise=true , address=45.91.116.9:4263 , piece_id=ZGWKEOEWLAADDOPVEKTXQEZQKJOKCO5LZ4PPK6FYF64AYG6DBIBA, seg_pos={0 0}, upload=11929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 26 05:41:00.792313144 (13h37m27.951078878s) [noise=true , address=104.36.82.9:4006 , piece_id=DXAOC2MPR5QWV3VSFEWSZ5QAYEHE4U4DZZTIWRBWJVXRVUNWGHJQ, seg_pos={0 0}, upload=11929]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: expected piece hash; context canceled; EOF Aug 26 05:41:00.792334874 (13h37m27.951100608s) [seg_pos={0 0}, upload=11929]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:41:00.792422904 (13h37m27.951188648s) [upload=11929]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:41:00.792428795 (13h37m27.951194539s) [upload=11929]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:41:19.983376712 (13h37m47.142142446s) [upload=11929]: Done uploading segments... Aug 26 05:41:19.983392883 (13h37m47.142158607s) [upload=11929]: Upload finished. err: <nil> Duration: 30.001113905s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [.........X.........X..X..XXX..................X........XX.X.] duration: 1.022242782s remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:41:20 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 1.022268292s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [X.....XX.........XXX.X.XXX..XX.......X.X.X...X.........XX..X] duration: 30.003904513s FAIL (2025-08-26 05:41:15 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 30.003938088s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [......XX..X........XXXX.X..X...X....X..X.X.XXX.X...X...XXX..] duration: 18.416289567s FAIL (2025-08-26 05:41:14 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A8C340A0206, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 18.416313433s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [...XX.........X...X...XXX.........X..X.....X.........XX.XX..] duration: 16.357923706s FAIL (2025-08-26 05:41:12 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 16.357954085s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 8 out of 60 is failed (threshold=3) [.......X...............XX.X.....X.........X..X..........X...] duration: 8.973846085s remote: "tcp:45.38.24.2:443" response headers: Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3A8584C361F7 X-Request-Id=ZpeTdY76RoJ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:41:09 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8584C361F7</RequestId><HostId></HostId></Error>" Duration: 8.973876423s edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 6 out of 60 is failed (threshold=6) [.....X................XX...........X...X.................X..] duration: 32.621242293s remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* X-Request-Id=DS54VnSBYuP Date=Tue, 26 Aug 2025 05:38:26 GMT Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:41:09 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 32.621284605s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [X...................X...X.X.....X.....X................X.XX.] duration: 20.845149935s FAIL (2025-08-26 05:41:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== ug 26 05:40:19.056939163 (13h36m46.215704897s) [upload=11918]: Done starting upload Aug 26 05:40:19.057012192 (13h36m46.215777926s) [upload=11918]: Uploading segments... Aug 26 05:40:19.057182428 (13h36m46.215948162s) [upload=11918]: Got next segment. Inline: false Aug 26 05:40:19.057226923 (13h36m46.215992667s) [upload=11918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:19.057233115 (13h36m46.215998849s) [upload=11918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:40:23.502627983 (13h36m50.661393717s) [seg_pos={0 0}, upload=11918]: Begin upload segment... Aug 26 05:40:23.502682537 (13h36m50.661448271s) [seg_pos={0 0}, upload=11918]: Done begin upload segment. Aug 26 05:40:23.502707064 (13h36m50.661472808s) [noise=true , address=104.36.82.5:4858 , piece_id=N23A4C6ZLYN22GHT3PI4G7MLTC7RPQUNJPX2PBCJ5V3VYLRD5UAQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.502777999 (13h36m50.661543773s) [noise=true , address=104.36.82.23:4447 , piece_id=XVPUXIOXBDK675IUWGUE73TML2TAVXQWUIVEYBSFWJIEC745OZSA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503118962 (13h36m50.661884696s) [noise=true , address=64.49.13.169:4710 , piece_id=WY2J6MCEVOOEE4DOO5NPXTGOSEAD4DBF6PHCZCWWXLOGFE5WTJCA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503338482 (13h36m50.662104226s) [noise=true , address=77.90.31.19:4856 , piece_id=UW5FADW2RN3DCVIZRANNQ7FJPMKMSJSYGG3YWM37ALAH23BNULLA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503427582 (13h36m50.662193317s) [noise=true , address=104.36.82.60:4348 , piece_id=ZJ5UN6PIVLBLJBBG7LH5I6TKVZKQQNMON5TASOH5NUJWNISEBAIA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503645159 (13h36m50.662410903s) [noise=true , address=79.127.201.215:4349 , piece_id=MSDHAY6MYVQ7RZAFXEFTL4WLZ7TQ3RZQBDG4A7JYWU3G55VBZB2A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503714150 (13h36m50.662479884s) [noise=true , address=77.90.31.10:4541 , piece_id=AC4ZXOUEIWEOZRICVCLDH6YFX5ZFLMVLVZ2WUNIO4IS7EWKQRGLQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503960742 (13h36m50.662726476s) [noise=true , address=45.91.116.5:4310 , piece_id=Y6WZUESALGWMEV7GQJNX3KBFVUICY3D4P3POXHEC52Y5Q6EHPTYA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504001300 (13h36m50.662767054s) [noise=true , address=77.90.31.12:4605 , piece_id=ZVYKBCV4Z3WBBAQ2KIMHYAGNSYGYXXOU5YV3EFG7HPT6ES6XUG2A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504277539 (13h36m50.663043283s) [noise=true , address=45.91.116.11:4139 , piece_id=WIA4ESSYFKI7UDDV33Y6KVLUK7G6CXMYOLSRWVJ32OUZUAYZQZAA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504348275 (13h36m50.663113999s) [noise=true , address=77.90.31.16:4757 , piece_id=7NZMGVI3C4WPYRLCEWMBA2CHUHTFE7EFCLCKQXLXJ444DD6S4ZRQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504612369 (13h36m50.663378103s) [noise=true , address=45.91.116.12:4236 , piece_id=KGIV6WPJBDTNYRIG7HMUHQGA3GWOQ7W5MUYQP3PWUTG27QFAL75Q, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504633861 (13h36m50.663399595s) [noise=true , address=45.91.116.9:4272 , piece_id=QHD2H6RCYRDBOI6VGSXNJS6BZH2BURGKIRB4UCZNQXK5WISDTDHQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504852048 (13h36m50.663617792s) [noise=true , address=104.36.82.11:4142 , piece_id=22YSQTTKXJHNE3AQ44A7NJGUA4DKSOVIZBOTMT3HJYDWCU3UCCPA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505075856 (13h36m50.66384159s) [noise=true , address=77.90.31.13:4634 , piece_id=GO5S4VSTUQRQ3IJ6OTSUDVHO4ST7LN4CZIYMUMUPTWC4Y62NXK4A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505287752 (13h36m50.664053486s) [noise=true , address=104.36.82.50:4027 , piece_id=R5HLBUJKURCYHQYXIBV3RYAFWQVLKVITTXWYXDUY5NGP5J2BQCSQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505492173 (13h36m50.664257907s) [noise=true , address=104.36.82.44:4874 , piece_id=364PSYMOKSTJVGMKFLV5L3435NVAH7Z3IYSQ6EAGHMDQTTVH6NGQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505693018 (13h36m50.664458752s) [noise=true , address=45.91.116.6:4002 , piece_id=T3DZ2WWRKNQJRAL6N2PU5ES5HMEZJ7PF2JWCUDA2D4MSB2IZYOEQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:24.620780439 (13h36m51.779546173s) [upload=11918]: Next returned nil segment Aug 26 05:40:24.620798893 (13h36m51.779564628s) [upload=11918]: Waiting for error group managing segments... Aug 26 05:40:24.902230365 (13h36m52.060996099s) [noise=true , address=64.49.13.169:4710 , piece_id=WY2J6MCEVOOEE4DOO5NPXTGOSEAD4DBF6PHCZCWWXLOGFE5WTJCA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.385848630 (13h36m52.544614364s) [noise=true , address=45.91.116.12:4236 , piece_id=KGIV6WPJBDTNYRIG7HMUHQGA3GWOQ7W5MUYQP3PWUTG27QFAL75Q, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.400066525 (13h36m52.558832259s) [noise=true , address=45.91.116.6:4002 , piece_id=T3DZ2WWRKNQJRAL6N2PU5ES5HMEZJ7PF2JWCUDA2D4MSB2IZYOEQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.401071398 (13h36m52.559837142s) [noise=true , address=45.91.116.11:4139 , piece_id=WIA4ESSYFKI7UDDV33Y6KVLUK7G6CXMYOLSRWVJ32OUZUAYZQZAA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.403842370 (13h36m52.562608104s) [noise=true , address=77.90.31.10:4541 , piece_id=AC4ZXOUEIWEOZRICVCLDH6YFX5ZFLMVLVZ2WUNIO4IS7EWKQRGLQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.417858378 (13h36m52.576624112s) [noise=true , address=104.36.82.50:4027 , piece_id=R5HLBUJKURCYHQYXIBV3RYAFWQVLKVITTXWYXDUY5NGP5J2BQCSQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.427439481 (13h36m52.586205215s) [noise=true , address=104.36.82.5:4858 , piece_id=N23A4C6ZLYN22GHT3PI4G7MLTC7RPQUNJPX2PBCJ5V3VYLRD5UAQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.438229877 (13h36m52.596995611s) [noise=true , address=104.36.82.11:4142 , piece_id=22YSQTTKXJHNE3AQ44A7NJGUA4DKSOVIZBOTMT3HJYDWCU3UCCPA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.442695412 (13h36m52.601461166s) [noise=true , address=77.90.31.19:4856 , piece_id=UW5FADW2RN3DCVIZRANNQ7FJPMKMSJSYGG3YWM37ALAH23BNULLA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.454484609 (13h36m52.613250333s) [noise=true , address=77.90.31.12:4605 , piece_id=ZVYKBCV4Z3WBBAQ2KIMHYAGNSYGYXXOU5YV3EFG7HPT6ES6XUG2A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.458672042 (13h36m52.617437776s) [noise=true , address=77.90.31.16:4757 , piece_id=7NZMGVI3C4WPYRLCEWMBA2CHUHTFE7EFCLCKQXLXJ444DD6S4ZRQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.465375397 (13h36m52.624141131s) [noise=true , address=104.36.82.23:4447 , piece_id=XVPUXIOXBDK675IUWGUE73TML2TAVXQWUIVEYBSFWJIEC745OZSA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.490568412 (13h36m52.649334146s) [noise=true , address=79.127.201.215:4349 , piece_id=MSDHAY6MYVQ7RZAFXEFTL4WLZ7TQ3RZQBDG4A7JYWU3G55VBZB2A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.492002425 (13h36m52.65076816s) [noise=true , address=45.91.116.5:4310 , piece_id=Y6WZUESALGWMEV7GQJNX3KBFVUICY3D4P3POXHEC52Y5Q6EHPTYA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.492030639 (13h36m52.650796363s) [seg_pos={0 0}, upload=11918]: Segment reached optimal threshold of 14 pieces. Aug 26 05:40:25.492179605 (13h36m52.650945349s) [noise=true , address=104.36.82.60:4348 , piece_id=ZJ5UN6PIVLBLJBBG7LH5I6TKVZKQQNMON5TASOH5NUJWNISEBAIA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492315124 (13h36m52.651080858s) [noise=true , address=45.91.116.9:4272 , piece_id=QHD2H6RCYRDBOI6VGSXNJS6BZH2BURGKIRB4UCZNQXK5WISDTDHQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492340662 (13h36m52.651106396s) [noise=true , address=104.36.82.44:4874 , piece_id=364PSYMOKSTJVGMKFLV5L3435NVAH7Z3IYSQ6EAGHMDQTTVH6NGQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492377373 (13h36m52.651143107s) [noise=true , address=77.90.31.13:4634 , piece_id=GO5S4VSTUQRQ3IJ6OTSUDVHO4ST7LN4CZIYMUMUPTWC4Y62NXK4A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492395517 (13h36m52.651161251s) [seg_pos={0 0}, upload=11918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:40:25.492459320 (13h36m52.651225054s) [upload=11918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:40:25.492464981 (13h36m52.651230715s) [upload=11918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:26.174442779 (13h36m53.333208513s) [upload=11918]: Done uploading segments... Aug 26 05:40:26.174468968 (13h36m53.333234712s) [upload=11918]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 20.845234516s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [....X.X...X.......XXX.X..XXX......X.X.XX.XX..XX.....X.....X.] duration: 15.633659547s FAIL (2025-08-26 05:40:54 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A955443D8AA, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 15.633690507s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [......................XX...X......X..........X........XX.XX.] duration: 8.186599176s FAIL (2025-08-26 05:40:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:40:22.385961260 (13h36m49.544726994s) [upload=11919]: Starting upload Aug 26 05:40:22.386018800 (13h36m49.544784534s) [upload=11919]: Done starting upload Aug 26 05:40:22.386072673 (13h36m49.544838407s) [upload=11919]: Uploading segments... Aug 26 05:40:22.386111086 (13h36m49.54487681s) [upload=11919]: Got next segment. Inline: true Aug 26 05:40:22.386115005 (13h36m49.544880739s) [upload=11919]: Waiting for error group managing segments... Aug 26 05:40:22.386155261 (13h36m49.544920995s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:22.386161824 (13h36m49.544927558s) [upload=11919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:22.386164499 (13h36m49.544930233s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:23.933385976 (13h36m51.09215171s) [upload=11919]: Done uploading segments... Aug 26 05:40:23.933404712 (13h36m51.092170446s) [upload=11919]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 8.186646708s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.....................X.XX........X........................X.] duration: 35.303621874s FAIL (2025-08-26 05:40:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 35.303633167s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [X...XX.........X...X...XXX.........X..X.....X.........XX.XX.] duration: 23.19491695s FAIL (2025-08-26 05:40:40 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 23.194944553s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [.XXX....XX........XX..XX.XXXX...XXX..X....X.XX........XX.XX.] duration: 2.744284258s FAIL (2025-08-26 05:40:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:40:02.848961511 (13h36m30.007727255s) [upload=11914]: Starting upload Aug 26 05:40:02.849061402 (13h36m30.007827136s) [upload=11914]: Done starting upload Aug 26 05:40:02.849152336 (13h36m30.00791808s) [upload=11914]: Uploading segments... Aug 26 05:40:02.849226467 (13h36m30.007992202s) [upload=11914]: Got next segment. Inline: true Aug 26 05:40:02.849233061 (13h36m30.007998805s) [upload=11914]: Waiting for error group managing segments... Aug 26 05:40:02.849309486 (13h36m30.00807523s) [upload=11914]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:02.849316060 (13h36m30.008081794s) [upload=11914]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:02.849319957 (13h36m30.008085701s) [upload=11914]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:32.190408479 (13h36m59.349174243s) [upload=11914]: Done uploading segments... Aug 26 05:40:32.190448245 (13h36m59.349213999s) [upload=11914]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.744330146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 24 out of 60 is failed (threshold=3) [..XXX....XX........XX..XX.XXXX...XXX..X....X.XX........XX.XX] duration: 30.001508153s FAIL (2025-08-26 05:40:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:40:02.848961511 (13h36m30.007727255s) [upload=11914]: Starting upload Aug 26 05:40:02.849061402 (13h36m30.007827136s) [upload=11914]: Done starting upload Aug 26 05:40:02.849152336 (13h36m30.00791808s) [upload=11914]: Uploading segments... Aug 26 05:40:02.849226467 (13h36m30.007992202s) [upload=11914]: Got next segment. Inline: true Aug 26 05:40:02.849233061 (13h36m30.007998805s) [upload=11914]: Waiting for error group managing segments... Aug 26 05:40:02.849309486 (13h36m30.00807523s) [upload=11914]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:02.849316060 (13h36m30.008081794s) [upload=11914]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:02.849319957 (13h36m30.008085701s) [upload=11914]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:32.190408479 (13h36m59.349174243s) [upload=11914]: Done uploading segments... Aug 26 05:40:32.190448245 (13h36m59.349213999s) [upload=11914]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001535965s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 8 out of 60 is failed (threshold=3) [........X...............XX.X.....X.........X..X..........X..] duration: 4.583184593s remote: "tcp:45.38.24.2:443" response headers: Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3A8584C361F7 X-Request-Id=ZpeTdY76RoJ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:40:29 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8584C361F7</RequestId><HostId></HostId></Error>" Duration: 4.58320912s edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [..........X.........X..X..XXX..................X........XX.X] duration: 608.932151ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:40:28 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 608.964604ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 9 out of 60 is failed (threshold=3) [.X...................X...X.X.....X.....X................X.XX] duration: 7.765277027s FAIL (2025-08-26 05:40:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== ug 26 05:40:19.056939163 (13h36m46.215704897s) [upload=11918]: Done starting upload Aug 26 05:40:19.057012192 (13h36m46.215777926s) [upload=11918]: Uploading segments... Aug 26 05:40:19.057182428 (13h36m46.215948162s) [upload=11918]: Got next segment. Inline: false Aug 26 05:40:19.057226923 (13h36m46.215992667s) [upload=11918]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:19.057233115 (13h36m46.215998849s) [upload=11918]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:40:23.502627983 (13h36m50.661393717s) [seg_pos={0 0}, upload=11918]: Begin upload segment... Aug 26 05:40:23.502682537 (13h36m50.661448271s) [seg_pos={0 0}, upload=11918]: Done begin upload segment. Aug 26 05:40:23.502707064 (13h36m50.661472808s) [noise=true , address=104.36.82.5:4858 , piece_id=N23A4C6ZLYN22GHT3PI4G7MLTC7RPQUNJPX2PBCJ5V3VYLRD5UAQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.502777999 (13h36m50.661543773s) [noise=true , address=104.36.82.23:4447 , piece_id=XVPUXIOXBDK675IUWGUE73TML2TAVXQWUIVEYBSFWJIEC745OZSA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503118962 (13h36m50.661884696s) [noise=true , address=64.49.13.169:4710 , piece_id=WY2J6MCEVOOEE4DOO5NPXTGOSEAD4DBF6PHCZCWWXLOGFE5WTJCA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503338482 (13h36m50.662104226s) [noise=true , address=77.90.31.19:4856 , piece_id=UW5FADW2RN3DCVIZRANNQ7FJPMKMSJSYGG3YWM37ALAH23BNULLA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503427582 (13h36m50.662193317s) [noise=true , address=104.36.82.60:4348 , piece_id=ZJ5UN6PIVLBLJBBG7LH5I6TKVZKQQNMON5TASOH5NUJWNISEBAIA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503645159 (13h36m50.662410903s) [noise=true , address=79.127.201.215:4349 , piece_id=MSDHAY6MYVQ7RZAFXEFTL4WLZ7TQ3RZQBDG4A7JYWU3G55VBZB2A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503714150 (13h36m50.662479884s) [noise=true , address=77.90.31.10:4541 , piece_id=AC4ZXOUEIWEOZRICVCLDH6YFX5ZFLMVLVZ2WUNIO4IS7EWKQRGLQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.503960742 (13h36m50.662726476s) [noise=true , address=45.91.116.5:4310 , piece_id=Y6WZUESALGWMEV7GQJNX3KBFVUICY3D4P3POXHEC52Y5Q6EHPTYA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504001300 (13h36m50.662767054s) [noise=true , address=77.90.31.12:4605 , piece_id=ZVYKBCV4Z3WBBAQ2KIMHYAGNSYGYXXOU5YV3EFG7HPT6ES6XUG2A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504277539 (13h36m50.663043283s) [noise=true , address=45.91.116.11:4139 , piece_id=WIA4ESSYFKI7UDDV33Y6KVLUK7G6CXMYOLSRWVJ32OUZUAYZQZAA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504348275 (13h36m50.663113999s) [noise=true , address=77.90.31.16:4757 , piece_id=7NZMGVI3C4WPYRLCEWMBA2CHUHTFE7EFCLCKQXLXJ444DD6S4ZRQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504612369 (13h36m50.663378103s) [noise=true , address=45.91.116.12:4236 , piece_id=KGIV6WPJBDTNYRIG7HMUHQGA3GWOQ7W5MUYQP3PWUTG27QFAL75Q, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504633861 (13h36m50.663399595s) [noise=true , address=45.91.116.9:4272 , piece_id=QHD2H6RCYRDBOI6VGSXNJS6BZH2BURGKIRB4UCZNQXK5WISDTDHQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.504852048 (13h36m50.663617792s) [noise=true , address=104.36.82.11:4142 , piece_id=22YSQTTKXJHNE3AQ44A7NJGUA4DKSOVIZBOTMT3HJYDWCU3UCCPA, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505075856 (13h36m50.66384159s) [noise=true , address=77.90.31.13:4634 , piece_id=GO5S4VSTUQRQ3IJ6OTSUDVHO4ST7LN4CZIYMUMUPTWC4Y62NXK4A, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505287752 (13h36m50.664053486s) [noise=true , address=104.36.82.50:4027 , piece_id=R5HLBUJKURCYHQYXIBV3RYAFWQVLKVITTXWYXDUY5NGP5J2BQCSQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505492173 (13h36m50.664257907s) [noise=true , address=104.36.82.44:4874 , piece_id=364PSYMOKSTJVGMKFLV5L3435NVAH7Z3IYSQ6EAGHMDQTTVH6NGQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:23.505693018 (13h36m50.664458752s) [noise=true , address=45.91.116.6:4002 , piece_id=T3DZ2WWRKNQJRAL6N2PU5ES5HMEZJ7PF2JWCUDA2D4MSB2IZYOEQ, seg_pos={0 0}, upload=11918]: Uploading piece... Aug 26 05:40:24.620780439 (13h36m51.779546173s) [upload=11918]: Next returned nil segment Aug 26 05:40:24.620798893 (13h36m51.779564628s) [upload=11918]: Waiting for error group managing segments... Aug 26 05:40:24.902230365 (13h36m52.060996099s) [noise=true , address=64.49.13.169:4710 , piece_id=WY2J6MCEVOOEE4DOO5NPXTGOSEAD4DBF6PHCZCWWXLOGFE5WTJCA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.385848630 (13h36m52.544614364s) [noise=true , address=45.91.116.12:4236 , piece_id=KGIV6WPJBDTNYRIG7HMUHQGA3GWOQ7W5MUYQP3PWUTG27QFAL75Q, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.400066525 (13h36m52.558832259s) [noise=true , address=45.91.116.6:4002 , piece_id=T3DZ2WWRKNQJRAL6N2PU5ES5HMEZJ7PF2JWCUDA2D4MSB2IZYOEQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.401071398 (13h36m52.559837142s) [noise=true , address=45.91.116.11:4139 , piece_id=WIA4ESSYFKI7UDDV33Y6KVLUK7G6CXMYOLSRWVJ32OUZUAYZQZAA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.403842370 (13h36m52.562608104s) [noise=true , address=77.90.31.10:4541 , piece_id=AC4ZXOUEIWEOZRICVCLDH6YFX5ZFLMVLVZ2WUNIO4IS7EWKQRGLQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.417858378 (13h36m52.576624112s) [noise=true , address=104.36.82.50:4027 , piece_id=R5HLBUJKURCYHQYXIBV3RYAFWQVLKVITTXWYXDUY5NGP5J2BQCSQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.427439481 (13h36m52.586205215s) [noise=true , address=104.36.82.5:4858 , piece_id=N23A4C6ZLYN22GHT3PI4G7MLTC7RPQUNJPX2PBCJ5V3VYLRD5UAQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.438229877 (13h36m52.596995611s) [noise=true , address=104.36.82.11:4142 , piece_id=22YSQTTKXJHNE3AQ44A7NJGUA4DKSOVIZBOTMT3HJYDWCU3UCCPA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.442695412 (13h36m52.601461166s) [noise=true , address=77.90.31.19:4856 , piece_id=UW5FADW2RN3DCVIZRANNQ7FJPMKMSJSYGG3YWM37ALAH23BNULLA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.454484609 (13h36m52.613250333s) [noise=true , address=77.90.31.12:4605 , piece_id=ZVYKBCV4Z3WBBAQ2KIMHYAGNSYGYXXOU5YV3EFG7HPT6ES6XUG2A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.458672042 (13h36m52.617437776s) [noise=true , address=77.90.31.16:4757 , piece_id=7NZMGVI3C4WPYRLCEWMBA2CHUHTFE7EFCLCKQXLXJ444DD6S4ZRQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.465375397 (13h36m52.624141131s) [noise=true , address=104.36.82.23:4447 , piece_id=XVPUXIOXBDK675IUWGUE73TML2TAVXQWUIVEYBSFWJIEC745OZSA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.490568412 (13h36m52.649334146s) [noise=true , address=79.127.201.215:4349 , piece_id=MSDHAY6MYVQ7RZAFXEFTL4WLZ7TQ3RZQBDG4A7JYWU3G55VBZB2A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.492002425 (13h36m52.65076816s) [noise=true , address=45.91.116.5:4310 , piece_id=Y6WZUESALGWMEV7GQJNX3KBFVUICY3D4P3POXHEC52Y5Q6EHPTYA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: <nil> Aug 26 05:40:25.492030639 (13h36m52.650796363s) [seg_pos={0 0}, upload=11918]: Segment reached optimal threshold of 14 pieces. Aug 26 05:40:25.492179605 (13h36m52.650945349s) [noise=true , address=104.36.82.60:4348 , piece_id=ZJ5UN6PIVLBLJBBG7LH5I6TKVZKQQNMON5TASOH5NUJWNISEBAIA, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ezboNAWPbaK9UDENYoBb3iHjNU2ahn7mikiteN7y3Aj6uw4Nx): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492315124 (13h36m52.651080858s) [noise=true , address=45.91.116.9:4272 , piece_id=QHD2H6RCYRDBOI6VGSXNJS6BZH2BURGKIRB4UCZNQXK5WISDTDHQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492340662 (13h36m52.651106396s) [noise=true , address=104.36.82.44:4874 , piece_id=364PSYMOKSTJVGMKFLV5L3435NVAH7Z3IYSQ6EAGHMDQTTVH6NGQ, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YApHMARaC7Wxeqopt72gpnyJS1xYBwhV7wbU9NJdR6W7hukyq): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492377373 (13h36m52.651143107s) [noise=true , address=77.90.31.13:4634 , piece_id=GO5S4VSTUQRQ3IJ6OTSUDVHO4ST7LN4CZIYMUMUPTWC4Y62NXK4A, seg_pos={0 0}, upload=11918]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1WwFU9dJW2LLdTQ64FLEpt4qyQLVuiEb8PmSLrUokav2HGVCw7): protocol: expected piece hash; context canceled; EOF Aug 26 05:40:25.492395517 (13h36m52.651161251s) [seg_pos={0 0}, upload=11918]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:40:25.492459320 (13h36m52.651225054s) [upload=11918]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:40:25.492464981 (13h36m52.651230715s) [upload=11918]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:26.174442779 (13h36m53.333208513s) [upload=11918]: Done uploading segments... Aug 26 05:40:26.174468968 (13h36m53.333234712s) [upload=11918]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 7.765301405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 9 out of 60 is failed (threshold=3) [.......................XX...X......X..........X........XX.XX] duration: 2.190238568s FAIL (2025-08-26 05:40:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:40:22.385961260 (13h36m49.544726994s) [upload=11919]: Starting upload Aug 26 05:40:22.386018800 (13h36m49.544784534s) [upload=11919]: Done starting upload Aug 26 05:40:22.386072673 (13h36m49.544838407s) [upload=11919]: Uploading segments... Aug 26 05:40:22.386111086 (13h36m49.54487681s) [upload=11919]: Got next segment. Inline: true Aug 26 05:40:22.386115005 (13h36m49.544880739s) [upload=11919]: Waiting for error group managing segments... Aug 26 05:40:22.386155261 (13h36m49.544920995s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:40:22.386161824 (13h36m49.544927558s) [upload=11919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:40:22.386164499 (13h36m49.544930233s) [upload=11919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:40:23.933385976 (13h36m51.09215171s) [upload=11919]: Done uploading segments... Aug 26 05:40:23.933404712 (13h36m51.092170446s) [upload=11919]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 2.190262844s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [......................X.XX........X........................X] duration: 1m0.0017613s FAIL (2025-08-26 05:40:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00177232s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 8 out of 60 is failed (threshold=3) [.........X...............XX.X.....X.........X..X..........X.] duration: 11.432750361s remote: "tcp:45.38.24.2:443" response headers: Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3A8584C361F7 X-Request-Id=ZpeTdY76RoJ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:40:10 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8584C361F7</RequestId><HostId></HostId></Error>" Duration: 11.432779278s edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [.......XX..X........XXXX.X..X...X....X..X.X.XXX.X...X...XXX.] duration: 24.335860017s FAIL (2025-08-26 05:40:09 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A8C340A0206, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 24.335881539s edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 6 out of 60 is failed (threshold=6) [......X................XX...........X...X.................X.] duration: 29.277870553s remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* X-Request-Id=DS54VnSBYuP Date=Tue, 26 Aug 2025 05:38:26 GMT Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:40:03 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 29.277902053s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 8 out of 60 is failed (threshold=3) [........................XX...X......X..........X........XX.X] duration: 27.982802925s FAIL (2025-08-26 05:39:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:39:30.443229745 (13h35m57.601995479s) [upload=11905]: Starting upload Aug 26 05:39:30.443295029 (13h35m57.602060763s) [upload=11905]: Done starting upload Aug 26 05:39:30.443373269 (13h35m57.602138993s) [upload=11905]: Uploading segments... Aug 26 05:39:30.443467380 (13h35m57.602233114s) [upload=11905]: Got next segment. Inline: true Aug 26 05:39:30.443472319 (13h35m57.602238043s) [upload=11905]: Waiting for error group managing segments... Aug 26 05:39:30.443531933 (13h35m57.602297657s) [upload=11905]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:39:30.443537183 (13h35m57.602302917s) [upload=11905]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:39:30.443542272 (13h35m57.602308006s) [upload=11905]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:42.190217622 (13h36m9.348983356s) [upload=11905]: Done uploading segments... Aug 26 05:39:42.190235937 (13h36m9.349001671s) [upload=11905]: Upload finished. err: <nil> Duration:* 27.982822423s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X...................X...X.X.....X.....X................X.X] duration: 10.541500199s FAIL (2025-08-26 05:39:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:39:44.985582794 (13h36m12.144348528s) [upload=11911]: Starting upload Aug 26 05:39:44.985708984 (13h36m12.144474719s) [upload=11911]: Done starting upload Aug 26 05:39:44.985839394 (13h36m12.144605138s) [upload=11911]: Uploading segments... Aug 26 05:39:44.986075176 (13h36m12.14484091s) [upload=11911]: Got next segment. Inline: false Aug 26 05:39:44.986099773 (13h36m12.144865507s) [upload=11911]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:39:44.986104041 (13h36m12.144869775s) [upload=11911]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:39:49.180120571 (13h36m16.338886305s) [seg_pos={0 0}, upload=11911]: Begin upload segment... Aug 26 05:39:49.180161801 (13h36m16.338927535s) [seg_pos={0 0}, upload=11911]: Done begin upload segment. Aug 26 05:39:49.180181458 (13h36m16.338947192s) [noise=true , address=77.90.31.15:4731 , piece_id=IWJRYRFALQWB7UY6WHY7M3CS5F6ZMITLJKUVGOYVDST52CQYFF4A, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.180577626 (13h36m16.33934336s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEWRCZGBHJFG22CIDUJXDNKBDNYOXJ57TSVJO2AAYTB26CFEH42Q, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.180839037 (13h36m16.339604771s) [noise=true , address=45.91.116.4:4950 , piece_id=ANDA2R5YEU5CBXQU3MOFSTSKXIFCKLALLFUAMRBVHNK6L5K7BTMQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.181086109 (13h36m16.339851843s) [noise=true , address=77.90.31.14:4694 , piece_id=LXATM4ZWPQ7PTYMGG2R2W27WZLY6LSNGT5E4S5GUNS7N7DGULX5Q, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.181326358 (13h36m16.340092082s) [noise=true , address=104.36.82.60:4365 , piece_id=AZXHYDVV76ERI4FXGBVA3IQBTCEEIELJVNACQQTSPHCCNP2QTDXQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.181528105 (13h36m16.340293839s) [noise=true , address=104.36.82.12:4181 , piece_id=OW3DOJGC4C4LB6MQOIA473LMC5FDL7GBLRWRLEEZ55DJ6QSDOYKA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.181744579 (13h36m16.340510303s) [noise=true , address=79.127.219.48:4105 , piece_id=YTVKH2JB67OLKXMEE36W4RLQW3W47357F2YA22LBGXHNE75CQVJQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.182013474 (13h36m16.340779208s) [noise=true , address=45.91.116.14:4206 , piece_id=T7UN7KRXE2ILSYCMSDSSKQEAQMGDPOFMUKQSLFUPOA3KKG44ZJXQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.182937050 (13h36m16.341702785s) [noise=true , address=45.91.116.7:4035 , piece_id=KYQ7URIUHWD4RRFSJMI3PMQ424NNMX2YCSFGKFMJI5PZA7OB7HLA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.183393063 (13h36m16.342158797s) [noise=true , address=79.127.219.51:4203 , piece_id=4XWPEYKZPLPBS3LF3UPZORN4MYJVYQNF4UP5UX5NN6RKNHM43QBQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.183657368 (13h36m16.342423103s) [noise=true , address=104.36.82.9:4038 , piece_id=QU5JF7JGMO4JQKAYHS4UCQV7OORUVGH27BQ5HINNA5ZDR7H5A3WA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.184022648 (13h36m16.342788382s) [noise=true , address=79.127.205.245:4231 , piece_id=HH2M6IIEEMAXVUHXSPHL757UJYCV6FIMVBDTTCIYSUY2TJGLMBBA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.184549085 (13h36m16.343314819s) [noise=true , address=79.127.201.215:4345 , piece_id=I5L5HAEZUGRJEKDJNFEQVI4ZTYZIPUTFAWH2NIDITVUN2UNR6EPQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.184937468 (13h36m16.343703192s) [noise=true , address=77.90.31.16:4760 , piece_id=P7LXYTJOT2YZVM56NBWZXBVC6YRCX7BUSAQB757L7PT5K2K3MBDA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.185144044 (13h36m16.343909768s) [noise=true , address=79.127.205.246:4254 , piece_id=EOBGD7ZMAF7DE4M7LNXJPAKBTBC3JHMRDXRWWHZCI36BGJTBFOMQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.185437224 (13h36m16.344202958s) [noise=true , address=45.91.116.5:4305 , piece_id=4JP5RSGP6YXJ756YPWGZ3YMJOPLH3SOO6GCD4TM6Y6KOCMJ7Q6AQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.185643539 (13h36m16.344409273s) [noise=true , address=45.91.116.13:4185 , piece_id=3KHRVIICBZLKTWAGZWSXGTKH6ZAYXOTMNVDX5K3NK6CCGWAEQ7UQ, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:49.185889720 (13h36m16.344655454s) [noise=true , address=104.36.82.23:4478 , piece_id=X5SKPF2DBSIA4HG5WGJBFNUQKI5UN6GGQB5FNDWMB5Z6JV4JWKHA, seg_pos={0 0}, upload=11911]: Uploading piece... Aug 26 05:39:50.304864319 (13h36m17.463630053s) [upload=11911]: Next returned nil segment Aug 26 05:39:50.304883836 (13h36m17.46364956s) [upload=11911]: Waiting for error group managing segments... Aug 26 05:39:50.591723726 (13h36m17.75048945s) [noise=true , address=79.127.205.246:4254 , piece_id=EOBGD7ZMAF7DE4M7LNXJPAKBTBC3JHMRDXRWWHZCI36BGJTBFOMQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:50.652358635 (13h36m17.811124369s) [noise=true , address=79.127.205.245:4231 , piece_id=HH2M6IIEEMAXVUHXSPHL757UJYCV6FIMVBDTTCIYSUY2TJGLMBBA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.069177517 (13h36m18.227943241s) [noise=true , address=45.91.116.4:4950 , piece_id=ANDA2R5YEU5CBXQU3MOFSTSKXIFCKLALLFUAMRBVHNK6L5K7BTMQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.073970057 (13h36m18.232735791s) [noise=true , address=45.91.116.7:4035 , piece_id=KYQ7URIUHWD4RRFSJMI3PMQ424NNMX2YCSFGKFMJI5PZA7OB7HLA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.074234053 (13h36m18.232999787s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEWRCZGBHJFG22CIDUJXDNKBDNYOXJ57TSVJO2AAYTB26CFEH42Q, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.076516520 (13h36m18.235282254s) [noise=true , address=77.90.31.15:4731 , piece_id=IWJRYRFALQWB7UY6WHY7M3CS5F6ZMITLJKUVGOYVDST52CQYFF4A, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.078184141 (13h36m18.236949875s) [noise=true , address=45.91.116.13:4185 , piece_id=3KHRVIICBZLKTWAGZWSXGTKH6ZAYXOTMNVDX5K3NK6CCGWAEQ7UQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.080743828 (13h36m18.239509562s) [noise=true , address=45.91.116.5:4305 , piece_id=4JP5RSGP6YXJ756YPWGZ3YMJOPLH3SOO6GCD4TM6Y6KOCMJ7Q6AQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.086597689 (13h36m18.245363423s) [noise=true , address=104.36.82.23:4478 , piece_id=X5SKPF2DBSIA4HG5WGJBFNUQKI5UN6GGQB5FNDWMB5Z6JV4JWKHA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.104887385 (13h36m18.263653119s) [noise=true , address=104.36.82.9:4038 , piece_id=QU5JF7JGMO4JQKAYHS4UCQV7OORUVGH27BQ5HINNA5ZDR7H5A3WA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.111029758 (13h36m18.269795492s) [noise=true , address=79.127.219.48:4105 , piece_id=YTVKH2JB67OLKXMEE36W4RLQW3W47357F2YA22LBGXHNE75CQVJQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.159344864 (13h36m18.318110598s) [noise=true , address=79.127.219.51:4203 , piece_id=4XWPEYKZPLPBS3LF3UPZORN4MYJVYQNF4UP5UX5NN6RKNHM43QBQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.163948052 (13h36m18.322713786s) [noise=true , address=45.91.116.14:4206 , piece_id=T7UN7KRXE2ILSYCMSDSSKQEAQMGDPOFMUKQSLFUPOA3KKG44ZJXQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.171935304 (13h36m18.330701038s) [noise=true , address=104.36.82.12:4181 , piece_id=OW3DOJGC4C4LB6MQOIA473LMC5FDL7GBLRWRLEEZ55DJ6QSDOYKA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.171945834 (13h36m18.330711568s) [seg_pos={0 0}, upload=11911]: Segment reached optimal threshold of 14 pieces. Aug 26 05:39:51.172026518 (13h36m18.330792252s) [noise=true , address=104.36.82.60:4365 , piece_id=AZXHYDVV76ERI4FXGBVA3IQBTCEEIELJVNACQQTSPHCCNP2QTDXQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122EX4PNuiRTCG2gbgbaL5dAtD8s5SN7ZesShk7NqUTGCZ3gA2a): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:51.172108064 (13h36m18.330873799s) [noise=true , address=79.127.201.215:4345 , piece_id=I5L5HAEZUGRJEKDJNFEQVI4ZTYZIPUTFAWH2NIDITVUN2UNR6EPQ, seg_pos={0 0}, upload=11911]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u4jspTRDLXgx4VABvYM3XdZ3fnEfvC7rSiAzKKccUy4k3y4F8): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:51.172161637 (13h36m18.330927371s) [noise=true , address=77.90.31.16:4760 , piece_id=P7LXYTJOT2YZVM56NBWZXBVC6YRCX7BUSAQB757L7PT5K2K3MBDA, seg_pos={0 0}, upload=11911]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1e6p1TD5CqUyWdZjkrxh2kRqMuyDiMUr9eimfeayDAKwhtBrZ8): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:51.172329187 (13h36m18.331094922s) [noise=true , address=77.90.31.14:4694 , piece_id=LXATM4ZWPQ7PTYMGG2R2W27WZLY6LSNGT5E4S5GUNS7N7DGULX5Q, seg_pos={0 0}, upload=11911]: Done uploading piece. err: <nil> Aug 26 05:39:51.172361900 (13h36m18.331127624s) [seg_pos={0 0}, upload=11911]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil> Aug 26 05:39:51.172437545 (13h36m18.331203279s) [upload=11911]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:39:51.172441923 (13h36m18.331207657s) [upload=11911]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:54.877789844 (13h36m22.036555568s) [upload=11911]: Done uploading segments... Aug 26 05:39:54.877845180 (13h36m22.036610904s) [upload=11911]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 10.541516219s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 20 out of 60 is failed (threshold=3) [.....X.X...X.......XXX.X..XXX......X.X.XX.XX..XX.....X.....X] duration: 9.162456644s FAIL (2025-08-26 05:39:54 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A955443D8AA, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 9.16247457s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [XX.....XX.........XXX.X.XXX..XX.......X.X.X...X.........XX..] duration: 14.155664298s FAIL (2025-08-26 05:39:48 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 14.155688015s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [X..............X.X...XXX....XXX.......XXX.X..X....X....X.XXX] duration: 7.644963891s FAIL (2025-08-26 05:39:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:39:40.499377327 (13h36m7.658143061s) [upload=11910]: Starting upload Aug 26 05:39:40.499459995 (13h36m7.658225729s) [upload=11910]: Done starting upload Aug 26 05:39:40.499546662 (13h36m7.658312386s) [upload=11910]: Uploading segments... Aug 26 05:39:40.499677071 (13h36m7.658442805s) [upload=11910]: Got next segment. Inline: false Aug 26 05:39:40.499710886 (13h36m7.65847662s) [upload=11910]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:39:40.499715585 (13h36m7.658481319s) [upload=11910]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:39:44.697690229 (13h36m11.856455953s) [seg_pos={0 0}, upload=11910]: Begin upload segment... Aug 26 05:39:44.697734654 (13h36m11.856500389s) [seg_pos={0 0}, upload=11910]: Done begin upload segment. Aug 26 05:39:44.697754332 (13h36m11.856520066s) [noise=true , address=45.91.116.9:4263 , piece_id=VT3QXO5AH4OVNDKKFFZPDUYBDV7B34P5NOFRJ3HCLECGYRUSS4KQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.697826911 (13h36m11.856592645s) [noise=true , address=45.91.116.11:4118 , piece_id=24IGT6MPXV4ZOCP3IJ2YFGAHMQ3NZ46Y7PWGDISEXW3ZJNQWRO4Q, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698143827 (13h36m11.856909552s) [noise=true , address=45.91.116.6:4973 , piece_id=K6X5LM3MSUFIKDPW6DGMUMFW65UK2E6LU6CQZ4PUX4DKOXBOFYLA, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698302752 (13h36m11.857068486s) [noise=true , address=104.36.82.7:4912 , piece_id=QXS2ZY2IFU6RMQ2Y74FQ534DX5HAPZHAVMQWWNVEYX4Z6KXRU6QA, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698405838 (13h36m11.857171562s) [noise=true , address=45.91.116.8:4045 , piece_id=R4LIA44WET3EFAG56W3D5QR7HHW3JJ6RRXGF6GWHL6J77EI6YS2Q, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698632032 (13h36m11.857397766s) [noise=true , address=104.36.82.8:4972 , piece_id=GUMEILIXVIQE2QJFB5V622S6E6YAGEULXZMEQVLOTSFSZFORRPMQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698685093 (13h36m11.857450827s) [noise=true , address=77.90.31.13:4626 , piece_id=WRAPAOCJ36B7WDUBJDW74JSJFDB7AEKE23OOU5EZW5GQRAFKR76A, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.698921144 (13h36m11.857686868s) [noise=true , address=77.90.31.21:4879 , piece_id=NOFQGVLRHJSX7JBLIGP5P7EDBMNH4HXUCFKIVVFGRSVOVS2L4OLQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.699171093 (13h36m11.857936827s) [noise=true , address=45.91.116.8:4048 , piece_id=WMOKXTL5D47N2RW7AHEN5OM3OFPAVDYEAEINC7MFXTIXXN5CU7LA, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.699410030 (13h36m11.858175764s) [noise=true , address=104.36.82.13:4186 , piece_id=4LOLD2EMW753RRTCDXPXWPMBZEFOZPEZG4OHVHKYTHIAGJGGQAJQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.700237924 (13h36m11.859003648s) [noise=true , address=45.91.116.5:4297 , piece_id=D6WDOUDSVPVNIVPYDQKITPMBRK5OOHNIBYS26DSHOB46XSVPVQ3A, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.702135035 (13h36m11.860900759s) [noise=true , address=45.91.116.4:4939 , piece_id=K7ZGAG5VFCXLPQSZLJ2KLJPNKGDHTP77ZAVJTFHALIMNHTO2XQPQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.702417665 (13h36m11.861183399s) [noise=true , address=45.91.116.11:4117 , piece_id=ALTDPAZ6N44YNJX7A6UBPBWI4MVTVHXE6CM5FKZ6SO6ZZ4IXTMHQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.702685909 (13h36m11.861451643s) [noise=true , address=77.90.31.19:4843 , piece_id=4XCZQL2JMMLWFVRY5B2CHINHHIUGDPNUCCG7QWMLPALDS5J7YRZQ, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.702955253 (13h36m11.861720987s) [noise=true , address=77.90.31.12:4590 , piece_id=A7Y5LUJEQL3S2B5SQWK7SMQLKXSQM4XZBW6HGY7SU24B2JG623BA, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:44.703231382 (13h36m11.861997117s) [noise=true , address=104.36.82.5:4839 , piece_id=PF4BMRDOINIS2SAJYGSJ62L3MLXL3WX2OKKDXC6RAWEC4ZKM2LVA, seg_pos={0 0}, upload=11910]: Uploading piece... Aug 26 05:39:46.409068063 (13h36m13.567833797s) [upload=11910]: Next returned nil segment Aug 26 05:39:46.409085216 (13h36m13.56785095s) [upload=11910]: Waiting for error group managing segments... Aug 26 05:39:46.721456258 (13h36m13.880221992s) [noise=true , address=45.91.116.11:4117 , piece_id=ALTDPAZ6N44YNJX7A6UBPBWI4MVTVHXE6CM5FKZ6SO6ZZ4IXTMHQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.724954381 (13h36m13.883720115s) [noise=true , address=45.91.116.9:4263 , piece_id=VT3QXO5AH4OVNDKKFFZPDUYBDV7B34P5NOFRJ3HCLECGYRUSS4KQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.728119908 (13h36m13.886885642s) [noise=true , address=45.91.116.5:4297 , piece_id=D6WDOUDSVPVNIVPYDQKITPMBRK5OOHNIBYS26DSHOB46XSVPVQ3A, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.732413944 (13h36m13.891179678s) [noise=true , address=45.91.116.6:4973 , piece_id=K6X5LM3MSUFIKDPW6DGMUMFW65UK2E6LU6CQZ4PUX4DKOXBOFYLA, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.737437737 (13h36m13.896203471s) [noise=true , address=77.90.31.19:4843 , piece_id=4XCZQL2JMMLWFVRY5B2CHINHHIUGDPNUCCG7QWMLPALDS5J7YRZQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.740511207 (13h36m13.899276941s) [noise=true , address=45.91.116.11:4118 , piece_id=24IGT6MPXV4ZOCP3IJ2YFGAHMQ3NZ46Y7PWGDISEXW3ZJNQWRO4Q, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.741319524 (13h36m13.900085258s) [noise=true , address=77.90.31.21:4879 , piece_id=NOFQGVLRHJSX7JBLIGP5P7EDBMNH4HXUCFKIVVFGRSVOVS2L4OLQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.741709701 (13h36m13.900475435s) [noise=true , address=45.91.116.8:4045 , piece_id=R4LIA44WET3EFAG56W3D5QR7HHW3JJ6RRXGF6GWHL6J77EI6YS2Q, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.743430413 (13h36m13.902196147s) [noise=true , address=45.91.116.4:4939 , piece_id=K7ZGAG5VFCXLPQSZLJ2KLJPNKGDHTP77ZAVJTFHALIMNHTO2XQPQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.763006801 (13h36m13.921772535s) [noise=true , address=104.36.82.8:4972 , piece_id=GUMEILIXVIQE2QJFB5V622S6E6YAGEULXZMEQVLOTSFSZFORRPMQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.776797969 (13h36m13.935563703s) [noise=true , address=104.36.82.5:4839 , piece_id=PF4BMRDOINIS2SAJYGSJ62L3MLXL3WX2OKKDXC6RAWEC4ZKM2LVA, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.806951865 (13h36m13.965717599s) [noise=true , address=77.90.31.13:4626 , piece_id=WRAPAOCJ36B7WDUBJDW74JSJFDB7AEKE23OOU5EZW5GQRAFKR76A, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.812042375 (13h36m13.97080811s) [noise=true , address=77.90.31.12:4590 , piece_id=A7Y5LUJEQL3S2B5SQWK7SMQLKXSQM4XZBW6HGY7SU24B2JG623BA, seg_pos={0 0}, upload=11910]: Done uploading piece. err: <nil> Aug 26 05:39:46.812073575 (13h36m13.970839309s) [seg_pos={0 0}, upload=11910]: Segment reached optimal threshold of 13 pieces. Aug 26 05:39:46.812273086 (13h36m13.971038821s) [noise=true , address=104.36.82.7:4912 , piece_id=QXS2ZY2IFU6RMQ2Y74FQ534DX5HAPZHAVMQWWNVEYX4Z6KXRU6QA, seg_pos={0 0}, upload=11910]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:46.812291542 (13h36m13.971057276s) [noise=true , address=45.91.116.8:4048 , piece_id=WMOKXTL5D47N2RW7AHEN5OM3OFPAVDYEAEINC7MFXTIXXN5CU7LA, seg_pos={0 0}, upload=11910]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:46.812368399 (13h36m13.971134133s) [noise=true , address=104.36.82.13:4186 , piece_id=4LOLD2EMW753RRTCDXPXWPMBZEFOZPEZG4OHVHKYTHIAGJGGQAJQ, seg_pos={0 0}, upload=11910]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:46.812394970 (13h36m13.971160704s) [seg_pos={0 0}, upload=11910]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:39:46.812468841 (13h36m13.971234575s) [upload=11910]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:39:46.812473090 (13h36m13.971238824s) [upload=11910]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:47.495307706 (13h36m14.65407344s) [upload=11910]: Done uploading segments... Aug 26 05:39:47.495334408 (13h36m14.654100132s) [upload=11910]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 7.644986776s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [.X...XX.........X...X...XXX.........X..X.....X.........XX.XX] duration: 7.106056735s FAIL (2025-08-26 05:39:46 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 7.106075892s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [...X...................X...X.X.....X.....X................X.] duration: 35.075355118s FAIL (2025-08-26 05:39:44 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:38:42.910990935 (13h35m10.069756669s) [upload=11894]: Starting upload Aug 26 05:38:42.911044087 (13h35m10.069809821s) [upload=11894]: Done starting upload Aug 26 05:38:42.911092780 (13h35m10.069858514s) [upload=11894]: Uploading segments... Aug 26 05:38:42.911309625 (13h35m10.070075359s) [upload=11894]: Got next segment. Inline: false Aug 26 05:38:42.911335535 (13h35m10.070101269s) [upload=11894]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:38:42.911339853 (13h35m10.070105587s) [upload=11894]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:38:47.635408313 (13h35m14.794174047s) [seg_pos={0 0}, upload=11894]: Begin upload segment... Aug 26 05:38:47.635450684 (13h35m14.794216419s) [seg_pos={0 0}, upload=11894]: Done begin upload segment. Aug 26 05:38:47.635469521 (13h35m14.794235255s) [noise=true , address=104.36.82.10:4088 , piece_id=ZGAO7PAPQ2U6SIB6OCMWJLWTWI6VP5LHQBY3MUPNSV5AYSMTIDTA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.635503936 (13h35m14.79426968s) [noise=true , address=156.146.43.225:4450 , piece_id=SBQXAHBRMNUEZJHFVW667UWEMMKTKG4ZMAY6ERTOEBOZ4JFTBESA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.635966361 (13h35m14.794732186s) [noise=true , address=45.91.116.14:4222 , piece_id=B6SBQY7R3ZLZVLD4LCXTF73MSPZNCTFIPQ2HJV37APAAEHZDYFTQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636301733 (13h35m14.795067467s) [noise=true , address=45.91.116.6:4001 , piece_id=4BJIHBDBUSK5B6LTUBAWDNT77WGWAPCR4ZEIYNCL6LKO3MDXFCCA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636762745 (13h35m14.795528479s) [noise=true , address=79.127.219.51:4206 , piece_id=IOQVRASLVI5LTVRKVGPDMNUEEJWYYOXYLGKHXRPITPWVIRVYZZSQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636817019 (13h35m14.795582753s) [noise=true , address=45.91.116.14:4214 , piece_id=6Y4UPWMUSZ4HZGUMVMOA5CTEJ7ACEHMRXMVGMYAAYL2VMDROUFNA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637050135 (13h35m14.795815869s) [noise=true , address=77.90.31.13:4625 , piece_id=NDPAGF2EEJZOC5GW4W6ULTQZDLJGIBUJP2H6RI25V3XZBUAG53OQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637382110 (13h35m14.796147854s) [noise=true , address=77.90.31.16:4765 , piece_id=ZFHPONXVV4AXHKYEWJISMIQHDP4FIMFTBP4UN5QIA6G3Q6AGHZOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637457404 (13h35m14.796223138s) [noise=true , address=104.36.82.23:4460 , piece_id=JNOPIFSEAFBGAY6AAKJYI7CI4OBON56FJKAM5XQ3TS6XVZDR6IRA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637722862 (13h35m14.796488586s) [noise=true , address=79.127.219.47:4069 , piece_id=5GGY7FJUN5Q535KGHORV3XH6HHZNTNKP2XWPWQZ6N5GTQE463NOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637893308 (13h35m14.796659042s) [noise=true , address=77.90.31.14:4686 , piece_id=3RM7UVWQOLGGLO4SUPTN75BO2ZEMZNRHEF6MRUEVOWTXV6DNSRPA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638139209 (13h35m14.796904933s) [noise=true , address=104.36.82.53:4127 , piece_id=7HKB6DQGVAON37X6LPNDUSNNGH6SVTMVL3D3LMZFQD3GWW6QD54Q, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638299796 (13h35m14.79706553s) [noise=true , address=79.127.201.215:4345 , piece_id=JBQUKCWPDPBJPYNWOGI7XQZQDANZJQQR5T7VQJDT7AFXAJ5IBUAA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638406540 (13h35m14.797172274s) [noise=true , address=77.90.31.10:4540 , piece_id=EKOFFHCDEVTW2JUGMG2YPSJ44SNOR6ZWCDBVY7EJZDJH7IMI4PRA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638621422 (13h35m14.797387156s) [noise=true , address=104.36.82.5:4848 , piece_id=FM6B5COMI2N3AXPZU4Y2KCF5Q2TZTOSR4XMVEPUPNBYQGBE3BNAA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638713628 (13h35m14.797479362s) [noise=true , address=104.36.82.60:4342 , piece_id=4IQ524A57NAY42ADICE52WIMAXY2MY63SLI5GVVFS4XGAU4XCDZA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638880968 (13h35m14.797646702s) [noise=true , address=104.36.82.44:4871 , piece_id=VO5KUOI7MTYWNF2LIRAWRR3G6OI44HIJQY5U5CCVETSIQBGD4P2A, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.639062055 (13h35m14.797827789s) [noise=true , address=104.36.82.14:4255 , piece_id=TJ3HJ34J73JMQRFNONCKX3OSR2NKKKCGVR5UZIQPEFIFLIA37UOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:49.126397455 (13h35m16.285163189s) [upload=11894]: Next returned nil segment Aug 26 05:38:49.126413214 (13h35m16.285178949s) [upload=11894]: Waiting for error group managing segments... Aug 26 05:38:49.528908280 (13h35m16.687674014s) [noise=true , address=45.91.116.6:4001 , piece_id=4BJIHBDBUSK5B6LTUBAWDNT77WGWAPCR4ZEIYNCL6LKO3MDXFCCA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.545057219 (13h35m16.703822953s) [noise=true , address=45.91.116.14:4214 , piece_id=6Y4UPWMUSZ4HZGUMVMOA5CTEJ7ACEHMRXMVGMYAAYL2VMDROUFNA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.546496994 (13h35m16.705262728s) [noise=true , address=45.91.116.14:4222 , piece_id=B6SBQY7R3ZLZVLD4LCXTF73MSPZNCTFIPQ2HJV37APAAEHZDYFTQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.546864336 (13h35m16.70563007s) [noise=true , address=104.36.82.44:4871 , piece_id=VO5KUOI7MTYWNF2LIRAWRR3G6OI44HIJQY5U5CCVETSIQBGD4P2A, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.548542247 (13h35m16.707307981s) [noise=true , address=104.36.82.60:4342 , piece_id=4IQ524A57NAY42ADICE52WIMAXY2MY63SLI5GVVFS4XGAU4XCDZA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.552854228 (13h35m16.711619962s) [noise=true , address=77.90.31.16:4765 , piece_id=ZFHPONXVV4AXHKYEWJISMIQHDP4FIMFTBP4UN5QIA6G3Q6AGHZOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.554932163 (13h35m16.713697897s) [noise=true , address=79.127.219.47:4069 , piece_id=5GGY7FJUN5Q535KGHORV3XH6HHZNTNKP2XWPWQZ6N5GTQE463NOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.572020511 (13h35m16.730786235s) [noise=true , address=104.36.82.5:4848 , piece_id=FM6B5COMI2N3AXPZU4Y2KCF5Q2TZTOSR4XMVEPUPNBYQGBE3BNAA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.584650486 (13h35m16.74341621s) [noise=true , address=77.90.31.14:4686 , piece_id=3RM7UVWQOLGGLO4SUPTN75BO2ZEMZNRHEF6MRUEVOWTXV6DNSRPA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.586859012 (13h35m16.745624747s) [noise=true , address=79.127.219.51:4206 , piece_id=IOQVRASLVI5LTVRKVGPDMNUEEJWYYOXYLGKHXRPITPWVIRVYZZSQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.587200435 (13h35m16.745966169s) [noise=true , address=156.146.43.225:4450 , piece_id=SBQXAHBRMNUEZJHFVW667UWEMMKTKG4ZMAY6ERTOEBOZ4JFTBESA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.592659150 (13h35m16.751424884s) [noise=true , address=104.36.82.53:4127 , piece_id=7HKB6DQGVAON37X6LPNDUSNNGH6SVTMVL3D3LMZFQD3GWW6QD54Q, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.597530090 (13h35m16.756295814s) [noise=true , address=104.36.82.14:4255 , piece_id=TJ3HJ34J73JMQRFNONCKX3OSR2NKKKCGVR5UZIQPEFIFLIA37UOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.603794988 (13h35m16.762560732s) [noise=true , address=77.90.31.13:4625 , piece_id=NDPAGF2EEJZOC5GW4W6ULTQZDLJGIBUJP2H6RI25V3XZBUAG53OQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.603829314 (13h35m16.762595048s) [seg_pos={0 0}, upload=11894]: Segment reached optimal threshold of 14 pieces. Aug 26 05:38:49.603944574 (13h35m16.762710308s) [noise=true , address=79.127.201.215:4345 , piece_id=JBQUKCWPDPBJPYNWOGI7XQZQDANZJQQR5T7VQJDT7AFXAJ5IBUAA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u4jspTRDLXgx4VABvYM3XdZ3fnEfvC7rSiAzKKccUy4k3y4F8): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.603993107 (13h35m16.762758841s) [noise=true , address=104.36.82.23:4460 , piece_id=JNOPIFSEAFBGAY6AAKJYI7CI4OBON56FJKAM5XQ3TS6XVZDR6IRA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PSoJT12SgP3n6ysTLMg1XwcdQmbqCPZYijha97LFC3LucEpFn): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604077869 (13h35m16.762843593s) [noise=true , address=77.90.31.10:4540 , piece_id=EKOFFHCDEVTW2JUGMG2YPSJ44SNOR6ZWCDBVY7EJZDJH7IMI4PRA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UUkKs1GGD9A6byJ1ZNZuZoX4kaa6zAdAKsaKs8bZ5A2jsxreA): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604133225 (13h35m16.762898959s) [noise=true , address=104.36.82.10:4088 , piece_id=ZGAO7PAPQ2U6SIB6OCMWJLWTWI6VP5LHQBY3MUPNSV5AYSMTIDTA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QbJar7xfiCubCHQ4hTTG4UYNVrxQig1mpvBKeXoEBUzbCHsJT): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604186106 (13h35m16.76295184s) [seg_pos={0 0}, upload=11894]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:38:49.604347786 (13h35m16.76311352s) [upload=11894]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:38:49.604352815 (13h35m16.763118549s) [upload=11894]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:05.018043410 (13h35m32.176809144s) [upload=11894]: Done uploading segments... Aug 26 05:39:05.018062537 (13h35m32.176828271s) [upload=11894]: Upload finished. err: <nil> Duration: 35.07539308s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........................XX...X......X..........X........XX.] duration: 31.173342338s FAIL (2025-08-26 05:39:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:38:14.663643455 (13h34m41.822409179s) [upload=11886]: Starting upload Aug 26 05:38:14.663699192 (13h34m41.822464926s) [upload=11886]: Done starting upload Aug 26 05:38:14.663747845 (13h34m41.822513579s) [upload=11886]: Uploading segments... Aug 26 05:38:14.663801708 (13h34m41.822567462s) [upload=11886]: Got next segment. Inline: true Aug 26 05:38:14.663808781 (13h34m41.822574576s) [upload=11886]: Waiting for error group managing segments... Aug 26 05:38:14.663853857 (13h34m41.822619591s) [upload=11886]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:38:14.663858306 (13h34m41.82262404s) [upload=11886]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:38:14.663860831 (13h34m41.822626565s) [upload=11886]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:38:15.306217077 (13h34m42.464982801s) [upload=11886]: Done uploading segments... Aug 26 05:38:15.306234520 (13h34m42.465000254s) [upload=11886]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 31.173370161s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.X..............X.X...XXX....XXX.......XXX.X..X....X....X.XX] duration: 4.041953433s FAIL (2025-08-26 05:39:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:39:20.212835015 (13h35m47.371600749s) [upload=11902]: Starting upload Aug 26 05:39:20.212914808 (13h35m47.371680542s) [upload=11902]: Done starting upload Aug 26 05:39:20.213021272 (13h35m47.371787006s) [upload=11902]: Uploading segments... Aug 26 05:39:20.213152373 (13h35m47.371918107s) [upload=11902]: Got next segment. Inline: false Aug 26 05:39:20.213184083 (13h35m47.371949817s) [upload=11902]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:39:20.213188411 (13h35m47.371954146s) [upload=11902]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:39:20.642987861 (13h35m47.801753596s) [seg_pos={0 0}, upload=11902]: Begin upload segment... Aug 26 05:39:20.643028639 (13h35m47.801794373s) [seg_pos={0 0}, upload=11902]: Done begin upload segment. Aug 26 05:39:20.643056242 (13h35m47.801821976s) [noise=true , address=77.90.31.13:4627 , piece_id=M2CN2R343CCKPSLUIEM6JDNUOHNURNMWHR2PG3GSZGPLUAKWAV7A, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643063406 (13h35m47.80182914s) [noise=true , address=77.90.31.14:4661 , piece_id=LZZMPP553BSNKCVGHSS4OOHOUFLFXQ5AKEIMUYAAIODUIUAYFK4A, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643413917 (13h35m47.802179651s) [noise=true , address=104.36.82.7:4912 , piece_id=ID44FIGIEW65BEGVO7NF3M3YU6WOBZJMAQ3345BT2VB5SJ3SLK2A, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643448152 (13h35m47.802213886s) [noise=true , address=45.91.116.6:4973 , piece_id=3SDUNMW7DMOGA42ZDOECJDY323TGRTQELOUBPYBE2UUBF4DXEXTQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643694704 (13h35m47.802460438s) [noise=true , address=77.90.31.13:4628 , piece_id=TP6BFOYP3XKSMPVVHCP5EA2RBIA3VUHHE4RLUGCPPPI6U744G2AQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643777913 (13h35m47.802543647s) [noise=true , address=77.90.31.14:4662 , piece_id=DXBVVO5RJ6V75XIRFGIUN2VBPFZGXVZC6JFB2LX3ZIUH7JXHUWHA, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.643972355 (13h35m47.802738079s) [noise=true , address=45.91.116.9:4261 , piece_id=MVKWSVMYAUYOEJTP4USJXQXHNWDYE2ARE3HC77AFKHQQG26DEAFQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644099228 (13h35m47.802864952s) [noise=true , address=77.90.31.19:4842 , piece_id=KTTBTNW5IE47G7MWYCPAEVRSYH3MOD4GRC4HSLC24477BO7AZMDA, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644280915 (13h35m47.803046639s) [noise=true , address=104.36.82.15:4259 , piece_id=6EJH5IMIOWZHIO6EO5R47GWQKK57HTSMIC7VANLQYATQPVSIBQJA, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644470118 (13h35m47.803235882s) [noise=true , address=45.91.116.13:4153 , piece_id=2QGASVZCPNLA7ASSLOMINZRIZCQLJQUYGLGOJUFQLZHBYUEGSQEQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644552455 (13h35m47.803318189s) [noise=true , address=77.90.31.21:4877 , piece_id=EYYYHEMFG4PRNQ6F45PI3IGOXU2CU6G376JHNWUQECWGSDGOZ64A, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644714535 (13h35m47.803480259s) [noise=true , address=77.90.31.10:4520 , piece_id=FGRCCFS73BJ3PX4CWCTSQR4JRZSBFLGLGFHH3B7WB4M6XXVJPCSQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644755253 (13h35m47.803520987s) [noise=true , address=45.91.116.10:4082 , piece_id=DYERLVFOHJ3XY7YODR2S5A2YWUV2FIFRCJUT47GOMYWMP5GJXUGA, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.644945507 (13h35m47.803711241s) [noise=true , address=77.90.31.15:4698 , piece_id=7CG2LIXDCF2POUQJWOODRB7D3TJH64KYUMGOWQEMGVPLPDZM2COQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.647511156 (13h35m47.80627689s) [noise=true , address=45.91.116.8:4047 , piece_id=T62RDVA77TEAZXMXBBZ7ZBK3CXK7DQ373N7TS37FFTN4XVF4SBBQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:20.650937112 (13h35m47.809702836s) [noise=true , address=77.90.31.10:4519 , piece_id=UR56ADFKCVAJDNHF5R63T4OXHBUEDNKKXM2H5RZZXSSQITK3D7OQ, seg_pos={0 0}, upload=11902]: Uploading piece... Aug 26 05:39:22.391093921 (13h35m49.549859645s) [upload=11902]: Next returned nil segment Aug 26 05:39:22.391110954 (13h35m49.549876688s) [upload=11902]: Waiting for error group managing segments... Aug 26 05:39:22.672594786 (13h35m49.83136052s) [noise=true , address=77.90.31.21:4877 , piece_id=EYYYHEMFG4PRNQ6F45PI3IGOXU2CU6G376JHNWUQECWGSDGOZ64A, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.673713276 (13h35m49.83247901s) [noise=true , address=77.90.31.10:4520 , piece_id=FGRCCFS73BJ3PX4CWCTSQR4JRZSBFLGLGFHH3B7WB4M6XXVJPCSQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.674990721 (13h35m49.833756455s) [noise=true , address=45.91.116.9:4261 , piece_id=MVKWSVMYAUYOEJTP4USJXQXHNWDYE2ARE3HC77AFKHQQG26DEAFQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.675107464 (13h35m49.833873218s) [noise=true , address=45.91.116.8:4047 , piece_id=T62RDVA77TEAZXMXBBZ7ZBK3CXK7DQ373N7TS37FFTN4XVF4SBBQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.677741273 (13h35m49.836507008s) [noise=true , address=45.91.116.13:4153 , piece_id=2QGASVZCPNLA7ASSLOMINZRIZCQLJQUYGLGOJUFQLZHBYUEGSQEQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.680826187 (13h35m49.839591921s) [noise=true , address=45.91.116.10:4082 , piece_id=DYERLVFOHJ3XY7YODR2S5A2YWUV2FIFRCJUT47GOMYWMP5GJXUGA, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.698458574 (13h35m49.857224308s) [noise=true , address=104.36.82.7:4912 , piece_id=ID44FIGIEW65BEGVO7NF3M3YU6WOBZJMAQ3345BT2VB5SJ3SLK2A, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.737390015 (13h35m49.896155749s) [noise=true , address=77.90.31.13:4627 , piece_id=M2CN2R343CCKPSLUIEM6JDNUOHNURNMWHR2PG3GSZGPLUAKWAV7A, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.758551245 (13h35m49.917316979s) [noise=true , address=77.90.31.19:4842 , piece_id=KTTBTNW5IE47G7MWYCPAEVRSYH3MOD4GRC4HSLC24477BO7AZMDA, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.765036965 (13h35m49.9238027s) [noise=true , address=45.91.116.6:4973 , piece_id=3SDUNMW7DMOGA42ZDOECJDY323TGRTQELOUBPYBE2UUBF4DXEXTQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.770552549 (13h35m49.929318283s) [noise=true , address=77.90.31.10:4519 , piece_id=UR56ADFKCVAJDNHF5R63T4OXHBUEDNKKXM2H5RZZXSSQITK3D7OQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.773316387 (13h35m49.932082121s) [noise=true , address=77.90.31.14:4661 , piece_id=LZZMPP553BSNKCVGHSS4OOHOUFLFXQ5AKEIMUYAAIODUIUAYFK4A, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.786253451 (13h35m49.945019185s) [noise=true , address=77.90.31.13:4628 , piece_id=TP6BFOYP3XKSMPVVHCP5EA2RBIA3VUHHE4RLUGCPPPI6U744G2AQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: <nil> Aug 26 05:39:22.786280343 (13h35m49.945046077s) [seg_pos={0 0}, upload=11902]: Segment reached optimal threshold of 13 pieces. Aug 26 05:39:22.786394932 (13h35m49.945160666s) [noise=true , address=77.90.31.14:4662 , piece_id=DXBVVO5RJ6V75XIRFGIUN2VBPFZGXVZC6JFB2LX3ZIUH7JXHUWHA, seg_pos={0 0}, upload=11902]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:22.786453604 (13h35m49.945219338s) [noise=true , address=77.90.31.15:4698 , piece_id=7CG2LIXDCF2POUQJWOODRB7D3TJH64KYUMGOWQEMGVPLPDZM2COQ, seg_pos={0 0}, upload=11902]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:22.786568273 (13h35m49.945334007s) [noise=true , address=104.36.82.15:4259 , piece_id=6EJH5IMIOWZHIO6EO5R47GWQKK57HTSMIC7VANLQYATQPVSIBQJA, seg_pos={0 0}, upload=11902]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Aug 26 05:39:22.786610054 (13h35m49.945375788s) [seg_pos={0 0}, upload=11902]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:39:22.786709824 (13h35m49.945475548s) [upload=11902]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:39:22.786714233 (13h35m49.945479967s) [upload=11902]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:23.602725410 (13h35m50.761491144s) [upload=11902]: Done uploading segments... Aug 26 05:39:23.602758334 (13h35m50.761524068s) [upload=11902]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 4.041974212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 23 out of 60 is failed (threshold=3) [...XXX....XX........XX..XX.XXXX...XXX..X....X.XX........XX.X] duration: 1.445442292s FAIL (2025-08-26 05:39:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:39:18.322535704 (13h35m45.481301438s) [upload=11901]: Starting upload Aug 26 05:39:18.322592452 (13h35m45.481358176s) [upload=11901]: Done starting upload Aug 26 05:39:18.322668788 (13h35m45.481434512s) [upload=11901]: Uploading segments... Aug 26 05:39:18.322701270 (13h35m45.481466994s) [upload=11901]: Got next segment. Inline: true Aug 26 05:39:18.322705128 (13h35m45.481470862s) [upload=11901]: Waiting for error group managing segments... Aug 26 05:39:18.322742208 (13h35m45.481507942s) [upload=11901]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:39:18.322746237 (13h35m45.481511971s) [upload=11901]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:39:18.322748802 (13h35m45.481514526s) [upload=11901]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:19.047110296 (13h35m46.20587603s) [upload=11901]: Done uploading segments... Aug 26 05:39:19.047132819 (13h35m46.205898554s) [upload=11901]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 1.445464204s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [...........X.........X..X..XXX..................X........XX.] duration: 556.421707ms remote: "tcp:207.211.208.129:443" response headers: Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=a6PxzJo9VCb Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3A8530B33F31 X-Xss-Protection=1; mode=block target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:39:19 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8530B33F31</RequestId><HostId></HostId></Error>" Duration: 556.473074ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [........XX..X........XXXX.X..X...X....X..X.X.XXX.X...X...XXX] duration: 7.546999145s FAIL (2025-08-26 05:39:14 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A8C340A0206, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 7.547018902s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......................X.XX........X........................] duration: 56.714972058s FAIL (2025-08-26 05:39:11 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F397B138BB539, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 56.715011755s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 7 out of 60 is failed (threshold=3) [....X...................X...X.X.....X.....X................X] duration: 26.394803969s FAIL (2025-08-26 05:39:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:38:42.910990935 (13h35m10.069756669s) [upload=11894]: Starting upload Aug 26 05:38:42.911044087 (13h35m10.069809821s) [upload=11894]: Done starting upload Aug 26 05:38:42.911092780 (13h35m10.069858514s) [upload=11894]: Uploading segments... Aug 26 05:38:42.911309625 (13h35m10.070075359s) [upload=11894]: Got next segment. Inline: false Aug 26 05:38:42.911335535 (13h35m10.070101269s) [upload=11894]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:38:42.911339853 (13h35m10.070105587s) [upload=11894]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:38:47.635408313 (13h35m14.794174047s) [seg_pos={0 0}, upload=11894]: Begin upload segment... Aug 26 05:38:47.635450684 (13h35m14.794216419s) [seg_pos={0 0}, upload=11894]: Done begin upload segment. Aug 26 05:38:47.635469521 (13h35m14.794235255s) [noise=true , address=104.36.82.10:4088 , piece_id=ZGAO7PAPQ2U6SIB6OCMWJLWTWI6VP5LHQBY3MUPNSV5AYSMTIDTA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.635503936 (13h35m14.79426968s) [noise=true , address=156.146.43.225:4450 , piece_id=SBQXAHBRMNUEZJHFVW667UWEMMKTKG4ZMAY6ERTOEBOZ4JFTBESA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.635966361 (13h35m14.794732186s) [noise=true , address=45.91.116.14:4222 , piece_id=B6SBQY7R3ZLZVLD4LCXTF73MSPZNCTFIPQ2HJV37APAAEHZDYFTQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636301733 (13h35m14.795067467s) [noise=true , address=45.91.116.6:4001 , piece_id=4BJIHBDBUSK5B6LTUBAWDNT77WGWAPCR4ZEIYNCL6LKO3MDXFCCA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636762745 (13h35m14.795528479s) [noise=true , address=79.127.219.51:4206 , piece_id=IOQVRASLVI5LTVRKVGPDMNUEEJWYYOXYLGKHXRPITPWVIRVYZZSQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.636817019 (13h35m14.795582753s) [noise=true , address=45.91.116.14:4214 , piece_id=6Y4UPWMUSZ4HZGUMVMOA5CTEJ7ACEHMRXMVGMYAAYL2VMDROUFNA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637050135 (13h35m14.795815869s) [noise=true , address=77.90.31.13:4625 , piece_id=NDPAGF2EEJZOC5GW4W6ULTQZDLJGIBUJP2H6RI25V3XZBUAG53OQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637382110 (13h35m14.796147854s) [noise=true , address=77.90.31.16:4765 , piece_id=ZFHPONXVV4AXHKYEWJISMIQHDP4FIMFTBP4UN5QIA6G3Q6AGHZOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637457404 (13h35m14.796223138s) [noise=true , address=104.36.82.23:4460 , piece_id=JNOPIFSEAFBGAY6AAKJYI7CI4OBON56FJKAM5XQ3TS6XVZDR6IRA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637722862 (13h35m14.796488586s) [noise=true , address=79.127.219.47:4069 , piece_id=5GGY7FJUN5Q535KGHORV3XH6HHZNTNKP2XWPWQZ6N5GTQE463NOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.637893308 (13h35m14.796659042s) [noise=true , address=77.90.31.14:4686 , piece_id=3RM7UVWQOLGGLO4SUPTN75BO2ZEMZNRHEF6MRUEVOWTXV6DNSRPA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638139209 (13h35m14.796904933s) [noise=true , address=104.36.82.53:4127 , piece_id=7HKB6DQGVAON37X6LPNDUSNNGH6SVTMVL3D3LMZFQD3GWW6QD54Q, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638299796 (13h35m14.79706553s) [noise=true , address=79.127.201.215:4345 , piece_id=JBQUKCWPDPBJPYNWOGI7XQZQDANZJQQR5T7VQJDT7AFXAJ5IBUAA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638406540 (13h35m14.797172274s) [noise=true , address=77.90.31.10:4540 , piece_id=EKOFFHCDEVTW2JUGMG2YPSJ44SNOR6ZWCDBVY7EJZDJH7IMI4PRA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638621422 (13h35m14.797387156s) [noise=true , address=104.36.82.5:4848 , piece_id=FM6B5COMI2N3AXPZU4Y2KCF5Q2TZTOSR4XMVEPUPNBYQGBE3BNAA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638713628 (13h35m14.797479362s) [noise=true , address=104.36.82.60:4342 , piece_id=4IQ524A57NAY42ADICE52WIMAXY2MY63SLI5GVVFS4XGAU4XCDZA, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.638880968 (13h35m14.797646702s) [noise=true , address=104.36.82.44:4871 , piece_id=VO5KUOI7MTYWNF2LIRAWRR3G6OI44HIJQY5U5CCVETSIQBGD4P2A, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:47.639062055 (13h35m14.797827789s) [noise=true , address=104.36.82.14:4255 , piece_id=TJ3HJ34J73JMQRFNONCKX3OSR2NKKKCGVR5UZIQPEFIFLIA37UOQ, seg_pos={0 0}, upload=11894]: Uploading piece... Aug 26 05:38:49.126397455 (13h35m16.285163189s) [upload=11894]: Next returned nil segment Aug 26 05:38:49.126413214 (13h35m16.285178949s) [upload=11894]: Waiting for error group managing segments... Aug 26 05:38:49.528908280 (13h35m16.687674014s) [noise=true , address=45.91.116.6:4001 , piece_id=4BJIHBDBUSK5B6LTUBAWDNT77WGWAPCR4ZEIYNCL6LKO3MDXFCCA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.545057219 (13h35m16.703822953s) [noise=true , address=45.91.116.14:4214 , piece_id=6Y4UPWMUSZ4HZGUMVMOA5CTEJ7ACEHMRXMVGMYAAYL2VMDROUFNA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.546496994 (13h35m16.705262728s) [noise=true , address=45.91.116.14:4222 , piece_id=B6SBQY7R3ZLZVLD4LCXTF73MSPZNCTFIPQ2HJV37APAAEHZDYFTQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.546864336 (13h35m16.70563007s) [noise=true , address=104.36.82.44:4871 , piece_id=VO5KUOI7MTYWNF2LIRAWRR3G6OI44HIJQY5U5CCVETSIQBGD4P2A, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.548542247 (13h35m16.707307981s) [noise=true , address=104.36.82.60:4342 , piece_id=4IQ524A57NAY42ADICE52WIMAXY2MY63SLI5GVVFS4XGAU4XCDZA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.552854228 (13h35m16.711619962s) [noise=true , address=77.90.31.16:4765 , piece_id=ZFHPONXVV4AXHKYEWJISMIQHDP4FIMFTBP4UN5QIA6G3Q6AGHZOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.554932163 (13h35m16.713697897s) [noise=true , address=79.127.219.47:4069 , piece_id=5GGY7FJUN5Q535KGHORV3XH6HHZNTNKP2XWPWQZ6N5GTQE463NOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.572020511 (13h35m16.730786235s) [noise=true , address=104.36.82.5:4848 , piece_id=FM6B5COMI2N3AXPZU4Y2KCF5Q2TZTOSR4XMVEPUPNBYQGBE3BNAA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.584650486 (13h35m16.74341621s) [noise=true , address=77.90.31.14:4686 , piece_id=3RM7UVWQOLGGLO4SUPTN75BO2ZEMZNRHEF6MRUEVOWTXV6DNSRPA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.586859012 (13h35m16.745624747s) [noise=true , address=79.127.219.51:4206 , piece_id=IOQVRASLVI5LTVRKVGPDMNUEEJWYYOXYLGKHXRPITPWVIRVYZZSQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.587200435 (13h35m16.745966169s) [noise=true , address=156.146.43.225:4450 , piece_id=SBQXAHBRMNUEZJHFVW667UWEMMKTKG4ZMAY6ERTOEBOZ4JFTBESA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.592659150 (13h35m16.751424884s) [noise=true , address=104.36.82.53:4127 , piece_id=7HKB6DQGVAON37X6LPNDUSNNGH6SVTMVL3D3LMZFQD3GWW6QD54Q, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.597530090 (13h35m16.756295814s) [noise=true , address=104.36.82.14:4255 , piece_id=TJ3HJ34J73JMQRFNONCKX3OSR2NKKKCGVR5UZIQPEFIFLIA37UOQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.603794988 (13h35m16.762560732s) [noise=true , address=77.90.31.13:4625 , piece_id=NDPAGF2EEJZOC5GW4W6ULTQZDLJGIBUJP2H6RI25V3XZBUAG53OQ, seg_pos={0 0}, upload=11894]: Done uploading piece. err: <nil> Aug 26 05:38:49.603829314 (13h35m16.762595048s) [seg_pos={0 0}, upload=11894]: Segment reached optimal threshold of 14 pieces. Aug 26 05:38:49.603944574 (13h35m16.762710308s) [noise=true , address=79.127.201.215:4345 , piece_id=JBQUKCWPDPBJPYNWOGI7XQZQDANZJQQR5T7VQJDT7AFXAJ5IBUAA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1u4jspTRDLXgx4VABvYM3XdZ3fnEfvC7rSiAzKKccUy4k3y4F8): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.603993107 (13h35m16.762758841s) [noise=true , address=104.36.82.23:4460 , piece_id=JNOPIFSEAFBGAY6AAKJYI7CI4OBON56FJKAM5XQ3TS6XVZDR6IRA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PSoJT12SgP3n6ysTLMg1XwcdQmbqCPZYijha97LFC3LucEpFn): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604077869 (13h35m16.762843593s) [noise=true , address=77.90.31.10:4540 , piece_id=EKOFFHCDEVTW2JUGMG2YPSJ44SNOR6ZWCDBVY7EJZDJH7IMI4PRA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UUkKs1GGD9A6byJ1ZNZuZoX4kaa6zAdAKsaKs8bZ5A2jsxreA): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604133225 (13h35m16.762898959s) [noise=true , address=104.36.82.10:4088 , piece_id=ZGAO7PAPQ2U6SIB6OCMWJLWTWI6VP5LHQBY3MUPNSV5AYSMTIDTA, seg_pos={0 0}, upload=11894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QbJar7xfiCubCHQ4hTTG4UYNVrxQig1mpvBKeXoEBUzbCHsJT): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:49.604186106 (13h35m16.76295184s) [seg_pos={0 0}, upload=11894]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:38:49.604347786 (13h35m16.76311352s) [upload=11894]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:38:49.604352815 (13h35m16.763118549s) [upload=11894]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:39:05.018043410 (13h35m32.176809144s) [upload=11894]: Done uploading segments... Aug 26 05:39:05.018062537 (13h35m32.176828271s) [upload=11894]: Upload finished. err: <nil> Duration: 26.394821101s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 20 out of 60 is failed (threshold=3) [XXX.....XX.........XXX.X.XXX..XX.......X.X.X...X.........XX.] duration: 19.570812396s FAIL (2025-08-26 05:38:56 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 19.570835571s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 19 out of 60 is failed (threshold=3) [......X.X...X.......XXX.X..XXX......X.X.XX.XX..XX.....X.....] duration: 17.523986346s FAIL (2025-08-26 05:38:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.524011985s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [....XXX....XX........XX..XX.XXXX...XXX..X....X.XX........XX.] duration: 20.357729906s FAIL (2025-08-26 05:38:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:38:00.802867376 (13h34m27.96163311s) [upload=11880]: Starting upload Aug 26 05:38:00.802919887 (13h34m27.961685611s) [upload=11880]: Done starting upload Aug 26 05:38:00.802969031 (13h34m27.961734765s) [upload=11880]: Uploading segments... Aug 26 05:38:00.803026891 (13h34m27.961792625s) [upload=11880]: Got next segment. Inline: true Aug 26 05:38:00.803031700 (13h34m27.961797425s) [upload=11880]: Waiting for error group managing segments... Aug 26 05:38:00.803070214 (13h34m27.961835948s) [upload=11880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:38:00.803075004 (13h34m27.961840728s) [upload=11880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:38:00.803077819 (13h34m27.961843553s) [upload=11880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:38:05.458685248 (13h34m32.617450982s) [upload=11880]: Done uploading segments... Aug 26 05:38:05.458703012 (13h34m32.617468746s) [upload=11880]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 20.357754833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [..X..............X.X...XXX....XXX.......XXX.X..X....X....X.X] duration: 3.84868398s FAIL (2025-08-26 05:38:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:38:44.461633437 (13h35m11.620399171s) [upload=11895]: Starting upload Aug 26 05:38:44.461690908 (13h35m11.620456632s) [upload=11895]: Done starting upload Aug 26 05:38:44.461761352 (13h35m11.620527086s) [upload=11895]: Uploading segments... Aug 26 05:38:44.461898575 (13h35m11.620664309s) [upload=11895]: Got next segment. Inline: false Aug 26 05:38:44.461934283 (13h35m11.620700007s) [upload=11895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:38:44.461938432 (13h35m11.620704166s) [upload=11895]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:38:44.815947262 (13h35m11.974712996s) [seg_pos={0 0}, upload=11895]: Begin upload segment... Aug 26 05:38:44.815988501 (13h35m11.974754235s) [seg_pos={0 0}, upload=11895]: Done begin upload segment. Aug 26 05:38:44.816010072 (13h35m11.974775806s) [noise=true , address=45.91.116.9:4261 , piece_id=HJDYI5ROW3NVPF7P66PPLUDPV3AIZKMCO5RCENCGZR5LEHNWJDTA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.816026884 (13h35m11.974792618s) [noise=true , address=45.91.116.10:4082 , piece_id=LXZ5H5ZYVIF6CPZR37HHL6WXOAE6ZUBB23IUHCAWACNDIPX2C52A, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.816437099 (13h35m11.975202833s) [noise=true , address=45.91.116.5:4298 , piece_id=75UU55QNCA43PKQBHWWHOGYZMNHNQVPXM5UOP7RTQQF57CYPXWKQ, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.816653453 (13h35m11.975419187s) [noise=true , address=77.90.31.10:4519 , piece_id=FHIBUY7NLDADVBSYMWA7QU3EWXB2MWQEXXMMEHFHOOIRYHHQN36Q, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.816715101 (13h35m11.975480835s) [noise=true , address=45.91.116.5:4297 , piece_id=GX6FPT4FE744HZVHK2FME3VHRX5R4QTQI5RZBOZX7OGXKORYLBNQ, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817024242 (13h35m11.975789976s) [noise=true , address=45.91.116.6:4973 , piece_id=GDYDIH3G3OUHS7W7SVRAY4VQD7CIOTGHLUJPOC2PDH4P25TQNSPA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817049360 (13h35m11.975815094s) [noise=true , address=77.90.31.15:4697 , piece_id=E4KPJ4F3KMZ5E27A6AEKMPRH7BSOHGCEXXHJISFPUJEAWMSNHDCA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817309118 (13h35m11.976074842s) [noise=true , address=77.90.31.18:4806 , piece_id=625FZZGXVKBMS2QH6CXFZRDL344P6BUOX7C2ZESL6A5CCRIVZJ3A, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817455958 (13h35m11.976221692s) [noise=true , address=45.91.116.6:4974 , piece_id=DUIBIQ2QT467UYMJQ6DCFJ2MWSEM7XWNMGOY24WOEZNMBB3Y3MHQ, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817590125 (13h35m11.976355869s) [noise=true , address=77.90.31.11:4554 , piece_id=24UI34TEBCXLMVT4I3UQFZMXAC7NQMYCTXQHMIK7MXVD2EFWFY3A, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817826257 (13h35m11.976591992s) [noise=true , address=45.91.116.10:4081 , piece_id=Y64KMNAF6M3RMPEG2Y4RX5B6QEN65O7JA346SFGQIYQPQWWONFWA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.817848560 (13h35m11.976614294s) [noise=true , address=45.91.116.6:4975 , piece_id=YN5KINXJAHPI5TMWYQBIXQLZD6XYUPC5PF5AEAWZWZZIWHVH37KQ, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.818037120 (13h35m11.976802854s) [noise=true , address=77.90.31.21:4877 , piece_id=TI3TGVLPCILSKUHBDJZXLIDLPVL7IO4HROS5OSN3HFI3GG55SC7Q, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.818189843 (13h35m11.976955567s) [noise=true , address=45.91.116.4:4940 , piece_id=GK3XNQD6Y2OOJ5V7IOOUVMRIO3W2F7FUPB2GZ7LW7I23IHOM7TRA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.818246812 (13h35m11.977012546s) [noise=true , address=45.91.116.8:4045 , piece_id=MYJXVQASIHGI3LS2SRUXYNM5ZP5OTOQ5SDVIIEAI5LBO5SITQMNQ, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:44.818566674 (13h35m11.977332408s) [noise=true , address=45.91.116.4:4939 , piece_id=GD6JTRTZYK4EYOBFEX5I7JJAGGYJLHFTAO4CJGXQP4LLUTCNTDGA, seg_pos={0 0}, upload=11895]: Uploading piece... Aug 26 05:38:46.536515680 (13h35m13.695281414s) [upload=11895]: Next returned nil segment Aug 26 05:38:46.536534356 (13h35m13.69530009s) [upload=11895]: Waiting for error group managing segments... Aug 26 05:38:46.836732065 (13h35m13.995497799s) [noise=true , address=77.90.31.21:4877 , piece_id=TI3TGVLPCILSKUHBDJZXLIDLPVL7IO4HROS5OSN3HFI3GG55SC7Q, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.842228121 (13h35m14.000993855s) [noise=true , address=45.91.116.8:4045 , piece_id=MYJXVQASIHGI3LS2SRUXYNM5ZP5OTOQ5SDVIIEAI5LBO5SITQMNQ, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.851662182 (13h35m14.010427916s) [noise=true , address=45.91.116.6:4974 , piece_id=DUIBIQ2QT467UYMJQ6DCFJ2MWSEM7XWNMGOY24WOEZNMBB3Y3MHQ, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.853106004 (13h35m14.011871738s) [noise=true , address=45.91.116.6:4973 , piece_id=GDYDIH3G3OUHS7W7SVRAY4VQD7CIOTGHLUJPOC2PDH4P25TQNSPA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.853406099 (13h35m14.012171833s) [noise=true , address=77.90.31.11:4554 , piece_id=24UI34TEBCXLMVT4I3UQFZMXAC7NQMYCTXQHMIK7MXVD2EFWFY3A, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.857604372 (13h35m14.016370116s) [noise=true , address=45.91.116.5:4298 , piece_id=75UU55QNCA43PKQBHWWHOGYZMNHNQVPXM5UOP7RTQQF57CYPXWKQ, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.879933847 (13h35m14.038699571s) [noise=true , address=45.91.116.4:4939 , piece_id=GD6JTRTZYK4EYOBFEX5I7JJAGGYJLHFTAO4CJGXQP4LLUTCNTDGA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.889897922 (13h35m14.048663656s) [noise=true , address=45.91.116.4:4940 , piece_id=GK3XNQD6Y2OOJ5V7IOOUVMRIO3W2F7FUPB2GZ7LW7I23IHOM7TRA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.890276156 (13h35m14.04904189s) [noise=true , address=77.90.31.10:4519 , piece_id=FHIBUY7NLDADVBSYMWA7QU3EWXB2MWQEXXMMEHFHOOIRYHHQN36Q, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.895259141 (13h35m14.054024875s) [noise=true , address=45.91.116.5:4297 , piece_id=GX6FPT4FE744HZVHK2FME3VHRX5R4QTQI5RZBOZX7OGXKORYLBNQ, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.922639109 (13h35m14.081404843s) [noise=true , address=77.90.31.18:4806 , piece_id=625FZZGXVKBMS2QH6CXFZRDL344P6BUOX7C2ZESL6A5CCRIVZJ3A, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.933853397 (13h35m14.092619121s) [noise=true , address=77.90.31.15:4697 , piece_id=E4KPJ4F3KMZ5E27A6AEKMPRH7BSOHGCEXXHJISFPUJEAWMSNHDCA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.944573679 (13h35m14.103339413s) [noise=true , address=45.91.116.10:4081 , piece_id=Y64KMNAF6M3RMPEG2Y4RX5B6QEN65O7JA346SFGQIYQPQWWONFWA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: <nil> Aug 26 05:38:46.944607194 (13h35m14.103372928s) [seg_pos={0 0}, upload=11895]: Segment reached optimal threshold of 13 pieces. Aug 26 05:38:46.944731631 (13h35m14.103497365s) [noise=true , address=45.91.116.6:4975 , piece_id=YN5KINXJAHPI5TMWYQBIXQLZD6XYUPC5PF5AEAWZWZZIWHVH37KQ, seg_pos={0 0}, upload=11895]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:46.944790744 (13h35m14.103556478s) [noise=true , address=45.91.116.10:4082 , piece_id=LXZ5H5ZYVIF6CPZR37HHL6WXOAE6ZUBB23IUHCAWACNDIPX2C52A, seg_pos={0 0}, upload=11895]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:46.944958305 (13h35m14.103724039s) [noise=true , address=45.91.116.9:4261 , piece_id=HJDYI5ROW3NVPF7P66PPLUDPV3AIZKMCO5RCENCGZR5LEHNWJDTA, seg_pos={0 0}, upload=11895]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi): protocol: expected piece hash; context canceled; EOF Aug 26 05:38:46.945030213 (13h35m14.103795947s) [seg_pos={0 0}, upload=11895]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:38:46.945147066 (13h35m14.1039128s) [upload=11895]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:38:46.945153750 (13h35m14.103919484s) [upload=11895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:38:47.666429940 (13h35m14.825195674s) [upload=11895]: Done uploading segments... Aug 26 05:38:47.666457132 (13h35m14.825222866s) [upload=11895]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 3.848696112s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 8 out of 60 is failed (threshold=3) [..........X...............XX.X.....X.........X..X..........X] duration: 842.733006ms remote: "tcp:45.38.24.2:443" response headers: Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3A8584C361F7 X-Request-Id=ZpeTdY76RoJ target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:38:45 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8584C361F7</RequestId><HostId></HostId></Error>" Duration: 842.758716ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.........XX..X........XXXX.X..X...X....X..X.X.XXX.X...X...XX] duration: 8.478467573s FAIL (2025-08-26 05:38:45 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3A8584670DB9, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 8.478488132s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [............X.........X..X..XXX..................X........XX] duration: 751.012608ms remote: "tcp:207.211.208.129:443" response headers: Date=Tue, 26 Aug 2025 05:38:44 GMT Accept-Ranges=bytes Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=a6PxzJo9VCb Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3A8530B33F31 X-Xss-Protection=1; mode=block target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:38:44 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A8530B33F31</RequestId><HostId></HostId></Error>" Duration: 751.032337ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...X..............X.X...XXX....XXX.......XXX.X..X....X....X.] duration: 25.38638182s FAIL (2025-08-26 05:38:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:37:44.186482127 (13h34m11.345247861s) [upload=11877]: Starting upload Aug 26 05:37:44.186585265 (13h34m11.345350999s) [upload=11877]: Done starting upload Aug 26 05:37:44.186704372 (13h34m11.345470107s) [upload=11877]: Uploading segments... Aug 26 05:37:44.186813431 (13h34m11.345579165s) [upload=11877]: Got next segment. Inline: false Aug 26 05:37:44.186852346 (13h34m11.34561807s) [upload=11877]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:37:44.186856674 (13h34m11.345622408s) [upload=11877]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:37:48.418871100 (13h34m15.577636834s) [seg_pos={0 0}, upload=11877]: Begin upload segment... Aug 26 05:37:48.418908922 (13h34m15.577674646s) [seg_pos={0 0}, upload=11877]: Done begin upload segment. Aug 26 05:37:48.418928840 (13h34m15.577694574s) [noise=true , address=45.91.116.11:4119 , piece_id=54UEAHFB3MCBVC2D6233QPVDZLOVH63HM7WZF5VWPFZKKOWS2STA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.418956263 (13h34m15.577721997s) [noise=true , address=77.90.31.18:4806 , piece_id=6O2TMEHHWUD2YZZXVEL66UFIVMF4J7RK5X4RZ6SGOLSXQV5DQO4Q, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.419280944 (13h34m15.578046678s) [noise=true , address=77.90.31.12:4590 , piece_id=LUW4OBT5N2Y7VZLSXP4QFUTH3LYOTPBRJFECYMPPVVROCH7XSZUQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.419304649 (13h34m15.578070373s) [noise=true , address=45.91.116.12:4227 , piece_id=WWU2X7DVI52AHMFTNKFCCQP4T5KVW7GBK6Z6BDJAQSOKA2IX6DLQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.419632105 (13h34m15.578397839s) [noise=true , address=45.91.116.14:4191 , piece_id=SV45SN45JLNHIHHDKWIADFNYO42DLZBCVTPHPSB2BTJ66IDKFQUA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.419641334 (13h34m15.578407068s) [noise=true , address=77.90.31.11:4553 , piece_id=45N6BPPGTQJP2AH3J4FVV3FGFKKQQREFVSBC56JDBTXJ6PC3W5SA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.419957147 (13h34m15.578722881s) [noise=true , address=45.91.116.4:4938 , piece_id=HODJF6D5DKDDB3CHHZ6NICZFPIF3QE526VHZOSEOTWT6RAYEO33Q, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420046458 (13h34m15.578812192s) [noise=true , address=45.91.116.10:4083 , piece_id=XCLND5J46PLK4OZMWFVLAJ2O3A7XLEVEE625PUP6OPIFHGJ3QWIA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420270447 (13h34m15.579036181s) [noise=true , address=77.90.31.19:4843 , piece_id=O7XKOVTL466YJFKJJIYR25TYKNGVDYG3TJ25RNQGO5N5MHDYGPNQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420339058 (13h34m15.579104782s) [noise=true , address=45.91.116.13:4155 , piece_id=MVFKRX7WDJTSVL4NGHEJKYEYB4YGJUKTTE57RQGAKZOYSRAAKESA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420546445 (13h34m15.579312169s) [noise=true , address=45.91.116.8:4048 , piece_id=TLNH7GJI3FDFENJFJQRNP7EZHPXEGAV2PK2TDPSDQRBXJBIR4AUA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420580240 (13h34m15.579345974s) [noise=true , address=45.91.116.8:4045 , piece_id=MISVNB2CTNTJPLBXLVPHEWF42TX364MVQGTCZL3SEEUJDOQ42QEQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420789771 (13h34m15.579555505s) [noise=true , address=45.91.116.4:4939 , piece_id=O7ST7QW3WKJ2YOCQSLK6GK647ED3TX6YKBRVOALGSTALWZGIIXFA, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.420803006 (13h34m15.57956873s) [noise=true , address=45.91.116.7:4011 , piece_id=PQV5MCDRT2MRFY6VBJ57DKNT575BYMGEHKNF3OCOIPV3Q6MYQPHQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.421023438 (13h34m15.579789182s) [noise=true , address=45.91.116.5:4299 , piece_id=W4RV5MHDDM3H6ASLELVJPEGOYGYECUYSEW2C2OINNCENI4HZTX5Q, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:48.421224462 (13h34m15.579990186s) [noise=true , address=45.91.116.14:4190 , piece_id=NZKKWDHWA4SBSJDDMGBYFBYW2DWJMA22ZG5I5YNWNSUL6YILXYJQ, seg_pos={0 0}, upload=11877]: Uploading piece... Aug 26 05:37:50.116483144 (13h34m17.275248878s) [upload=11877]: Next returned nil segment Aug 26 05:37:50.116499235 (13h34m17.275264969s) [upload=11877]: Waiting for error group managing segments... Aug 26 05:37:50.428703487 (13h34m17.587469221s) [noise=true , address=45.91.116.14:4190 , piece_id=NZKKWDHWA4SBSJDDMGBYFBYW2DWJMA22ZG5I5YNWNSUL6YILXYJQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.431447978 (13h34m17.590213712s) [noise=true , address=45.91.116.8:4048 , piece_id=TLNH7GJI3FDFENJFJQRNP7EZHPXEGAV2PK2TDPSDQRBXJBIR4AUA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.437324383 (13h34m17.596090117s) [noise=true , address=45.91.116.4:4938 , piece_id=HODJF6D5DKDDB3CHHZ6NICZFPIF3QE526VHZOSEOTWT6RAYEO33Q, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.440601483 (13h34m17.599367217s) [noise=true , address=45.91.116.10:4083 , piece_id=XCLND5J46PLK4OZMWFVLAJ2O3A7XLEVEE625PUP6OPIFHGJ3QWIA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.441050291 (13h34m17.599816025s) [noise=true , address=45.91.116.8:4045 , piece_id=MISVNB2CTNTJPLBXLVPHEWF42TX364MVQGTCZL3SEEUJDOQ42QEQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.442676343 (13h34m17.601442077s) [noise=true , address=45.91.116.14:4191 , piece_id=SV45SN45JLNHIHHDKWIADFNYO42DLZBCVTPHPSB2BTJ66IDKFQUA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.449463840 (13h34m17.608229574s) [noise=true , address=45.91.116.11:4119 , piece_id=54UEAHFB3MCBVC2D6233QPVDZLOVH63HM7WZF5VWPFZKKOWS2STA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.451794599 (13h34m17.610560323s) [noise=true , address=45.91.116.5:4299 , piece_id=W4RV5MHDDM3H6ASLELVJPEGOYGYECUYSEW2C2OINNCENI4HZTX5Q, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.455023207 (13h34m17.613788941s) [noise=true , address=45.91.116.13:4155 , piece_id=MVFKRX7WDJTSVL4NGHEJKYEYB4YGJUKTTE57RQGAKZOYSRAAKESA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.490929590 (13h34m17.649695324s) [noise=true , address=77.90.31.11:4553 , piece_id=45N6BPPGTQJP2AH3J4FVV3FGFKKQQREFVSBC56JDBTXJ6PC3W5SA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.512307836 (13h34m17.67107357s) [noise=true , address=77.90.31.18:4806 , piece_id=6O2TMEHHWUD2YZZXVEL66UFIVMF4J7RK5X4RZ6SGOLSXQV5DQO4Q, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.525482514 (13h34m17.684248249s) [noise=true , address=77.90.31.19:4843 , piece_id=O7XKOVTL466YJFKJJIYR25TYKNGVDYG3TJ25RNQGO5N5MHDYGPNQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.531445353 (13h34m17.690211087s) [noise=true , address=45.91.116.4:4939 , piece_id=O7ST7QW3WKJ2YOCQSLK6GK647ED3TX6YKBRVOALGSTALWZGIIXFA, seg_pos={0 0}, upload=11877]: Done uploading piece. err: <nil> Aug 26 05:37:50.531464029 (13h34m17.690229763s) [seg_pos={0 0}, upload=11877]: Segment reached optimal threshold of 13 pieces. Aug 26 05:37:50.531658291 (13h34m17.690424025s) [noise=true , address=77.90.31.12:4590 , piece_id=LUW4OBT5N2Y7VZLSXP4QFUTH3LYOTPBRJFECYMPPVVROCH7XSZUQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Aug 26 05:37:50.531678931 (13h34m17.690444655s) [noise=true , address=45.91.116.7:4011 , piece_id=PQV5MCDRT2MRFY6VBJ57DKNT575BYMGEHKNF3OCOIPV3Q6MYQPHQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf): protocol: expected piece hash; context canceled; EOF Aug 26 05:37:50.531771097 (13h34m17.690536871s) [noise=true , address=45.91.116.12:4227 , piece_id=WWU2X7DVI52AHMFTNKFCCQP4T5KVW7GBK6Z6BDJAQSOKA2IX6DLQ, seg_pos={0 0}, upload=11877]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:50718->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:50718->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50718->45.91.116.12:4227: use of closed network connection Aug 26 05:37:50.531802417 (13h34m17.690568141s) [seg_pos={0 0}, upload=11877]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:37:50.531865227 (13h34m17.690630961s) [upload=11877]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:37:50.531869235 (13h34m17.690634969s) [upload=11877]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:38:06.684728723 (13h34m33.843494457s) [upload=11877]: Done uploading segments... Aug 26 05:38:06.684746988 (13h34m33.843512712s) [upload=11877]: Upload finished. err: <nil> Duration: 25.386409823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X...................X...X.X.....X.....X................] duration: 58.722959014s FAIL (2025-08-26 05:38:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== ot next segment. Inline: false Aug 26 05:25:44.074200758 (13h22m11.232966492s) [upload=11711]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:25:44.074204846 (13h22m11.23297058s) [upload=11711]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:25:44.592716036 (13h22m11.75148178s) [seg_pos={0 0}, upload=11711]: Begin upload segment... Aug 26 05:25:44.592784087 (13h22m11.751549841s) [seg_pos={0 0}, upload=11711]: Done begin upload segment. Aug 26 05:25:44.592817892 (13h22m11.751583636s) [noise=true , address=45.91.116.5:4328 , piece_id=MMYZZAONFAMVHYLEYD6LFN6CJ763L6H742HM4PIBHXPO2JCFYXEA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.593368886 (13h22m11.75213462s) [noise=true , address=104.36.82.5:4864 , piece_id=ORHNRUFWDNB7CS2NOIL6JZZGOJ5QYLS3KR52U2NY3JCITP4AXONQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.593794320 (13h22m11.752560084s) [noise=true , address=77.90.31.19:4845 , piece_id=J2T7RSVVC2W6AG66EB6SKNX6LAU3F657S6FV3MN62JVH4QZSGWOQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.593999953 (13h22m11.752765687s) [noise=true , address=77.90.31.10:4533 , piece_id=A53K5G7CDQAQIU2BZ226EHH5Z4AL6UQW73VNC4Y2IS5UDXVDFWBA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.594218201 (13h22m11.752983945s) [noise=true , address=79.127.219.47:4038 , piece_id=UUJXRFTDE46AMX6XNIDJTGWKHE7ETJMCEDILSOWTAL4MILGY4JZQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.594432291 (13h22m11.753198025s) [noise=true , address=104.36.82.60:4351 , piece_id=QGLP6X5QPJ3LNUNTI22HVY7JFM62LLJQ2GCLFYF2RT2YYI74Q24A, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.594611744 (13h22m11.753377479s) [noise=true , address=104.36.82.44:4889 , piece_id=OYA7EVSJVKMMBHNYYGU6JWGEBI4WDE647TXOZ2BU4SPE6NQWWHAQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.594825723 (13h22m11.753591467s) [noise=true , address=79.127.201.215:4351 , piece_id=4W2NWWON2XBQFUPXUMA7BUG4X6CEA6MUKSHSLECJSXTNCI4JYQOQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.595010797 (13h22m11.753776531s) [noise=true , address=45.91.116.7:4030 , piece_id=XQNGDK3U77GL7BJPNYR44OXTHMWJCP2TXQ57WDIECEN7D3JSRWPQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.595191714 (13h22m11.753957448s) [noise=true , address=104.36.82.53:4145 , piece_id=JUPMIN3ZJFQLC6DJYWVIU4PMUPAQIOTLIS3LQFTFY5NV27SJFI6A, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.595537094 (13h22m11.754302828s) [noise=true , address=79.127.205.245:4249 , piece_id=JPOC27ZEVVB2HF62IMTULL2O5V6STD7UPOLX2MJ4LS7OXQ25DV2A, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.596097718 (13h22m11.754863452s) [noise=true , address=104.36.82.50:4032 , piece_id=LJMRUDRZXKLAVKA4S4XDT53CH343XP3XWJSZH5NDPOKGDAOBXMBA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.596498404 (13h22m11.755264138s) [noise=true , address=104.36.82.9:4040 , piece_id=UV743SWMBJ5PM5YZK6QDKPNGKQMT42SZ5Y7QZVRMBOZLLCDP2NOQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.596758792 (13h22m11.755524526s) [noise=true , address=77.90.31.14:4679 , piece_id=A6RXZWR2H4NKLGQSE5P3TMKWX54MJCZVIATXNQJWIVCSSTGYSSSQ, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.597051583 (13h22m11.755817317s) [noise=true , address=104.36.82.14:4237 , piece_id=TFWHMVZERVLDOCAKTVEK3G7A7JUM74WJUHRJQPP3DUVVLELRVLFA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.597294878 (13h22m11.756060612s) [noise=true , address=104.36.82.23:4448 , piece_id=KQM6XZNS5F3I2T24IA6G72IMXRNLXH6USJWHPEJRUZWBJVEUNKRA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.597522464 (13h22m11.756288198s) [noise=true , address=45.91.116.4:4962 , piece_id=A33F3SY46MASMYHN3N23DYBHYLYBIF27DNMRQ2UGLOYSM3V7FDIA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:44.597760348 (13h22m11.756526082s) [noise=true , address=104.36.82.10:4108 , piece_id=WQV5ERSKBZK6YDLTCRT4YL33RI7H7LBY6ZEBHOEBRFKPMZA2DHEA, seg_pos={0 0}, upload=11711]: Uploading piece... Aug 26 05:25:45.766739127 (13h22m12.925504861s) [upload=11711]: Next returned nil segment Aug 26 05:25:45.766759336 (13h22m12.92552507s) [upload=11711]: Waiting for error group managing segments... Aug 26 05:25:46.064800158 (13h22m13.223565892s) [noise=true , address=79.127.205.245:4249 , piece_id=JPOC27ZEVVB2HF62IMTULL2O5V6STD7UPOLX2MJ4LS7OXQ25DV2A, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.484860994 (13h22m13.643626728s) [noise=true , address=79.127.219.47:4038 , piece_id=UUJXRFTDE46AMX6XNIDJTGWKHE7ETJMCEDILSOWTAL4MILGY4JZQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.492359491 (13h22m13.651125225s) [noise=true , address=77.90.31.10:4533 , piece_id=A53K5G7CDQAQIU2BZ226EHH5Z4AL6UQW73VNC4Y2IS5UDXVDFWBA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.504809854 (13h22m13.663575578s) [noise=true , address=45.91.116.5:4328 , piece_id=MMYZZAONFAMVHYLEYD6LFN6CJ763L6H742HM4PIBHXPO2JCFYXEA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.526343026 (13h22m13.68510876s) [noise=true , address=45.91.116.4:4962 , piece_id=A33F3SY46MASMYHN3N23DYBHYLYBIF27DNMRQ2UGLOYSM3V7FDIA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.539564644 (13h22m13.698330398s) [noise=true , address=104.36.82.10:4108 , piece_id=WQV5ERSKBZK6YDLTCRT4YL33RI7H7LBY6ZEBHOEBRFKPMZA2DHEA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.549750103 (13h22m13.708515838s) [noise=true , address=104.36.82.14:4237 , piece_id=TFWHMVZERVLDOCAKTVEK3G7A7JUM74WJUHRJQPP3DUVVLELRVLFA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.562075396 (13h22m13.72084113s) [noise=true , address=77.90.31.19:4845 , piece_id=J2T7RSVVC2W6AG66EB6SKNX6LAU3F657S6FV3MN62JVH4QZSGWOQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.562335645 (13h22m13.721101379s) [noise=true , address=104.36.82.60:4351 , piece_id=QGLP6X5QPJ3LNUNTI22HVY7JFM62LLJQ2GCLFYF2RT2YYI74Q24A, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.564779851 (13h22m13.723545595s) [noise=true , address=104.36.82.53:4145 , piece_id=JUPMIN3ZJFQLC6DJYWVIU4PMUPAQIOTLIS3LQFTFY5NV27SJFI6A, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.568767681 (13h22m13.727533415s) [noise=true , address=79.127.201.215:4351 , piece_id=4W2NWWON2XBQFUPXUMA7BUG4X6CEA6MUKSHSLECJSXTNCI4JYQOQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.580552260 (13h22m13.739317994s) [noise=true , address=104.36.82.44:4889 , piece_id=OYA7EVSJVKMMBHNYYGU6JWGEBI4WDE647TXOZ2BU4SPE6NQWWHAQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.583750510 (13h22m13.742516244s) [noise=true , address=45.91.116.7:4030 , piece_id=XQNGDK3U77GL7BJPNYR44OXTHMWJCP2TXQ57WDIECEN7D3JSRWPQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.586524167 (13h22m13.745289901s) [noise=true , address=104.36.82.50:4032 , piece_id=LJMRUDRZXKLAVKA4S4XDT53CH343XP3XWJSZH5NDPOKGDAOBXMBA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: <nil> Aug 26 05:25:46.586539386 (13h22m13.74530511s) [seg_pos={0 0}, upload=11711]: Segment reached optimal threshold of 14 pieces. Aug 26 05:25:46.586653545 (13h22m13.745419279s) [noise=true , address=104.36.82.23:4448 , piece_id=KQM6XZNS5F3I2T24IA6G72IMXRNLXH6USJWHPEJRUZWBJVEUNKRA, seg_pos={0 0}, upload=11711]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i): protocol: expected piece hash; context canceled; EOF Aug 26 05:25:46.586778072 (13h22m13.745543806s) [noise=true , address=104.36.82.9:4040 , piece_id=UV743SWMBJ5PM5YZK6QDKPNGKQMT42SZ5Y7QZVRMBOZLLCDP2NOQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12reXxBn59mwVQqkszyY4X6NHrytLDAb4zQZQditjmgGqpMDabZ): protocol: expected piece hash; context canceled; EOF Aug 26 05:25:46.586795446 (13h22m13.74556118s) [noise=true , address=77.90.31.14:4679 , piece_id=A6RXZWR2H4NKLGQSE5P3TMKWX54MJCZVIATXNQJWIVCSSTGYSSSQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1k29MGpez3ByMHnG8qGBm9iPwo3VrKUGDHpz9CqGEGNn47rxDx): protocol: expected piece hash; context canceled; EOF Aug 26 05:25:46.586865761 (13h22m13.745631495s) [noise=true , address=104.36.82.5:4864 , piece_id=ORHNRUFWDNB7CS2NOIL6JZZGOJ5QYLS3KR52U2NY3JCITP4AXONQ, seg_pos={0 0}, upload=11711]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXH8TYK6Qfma867943QYYXycyL6osQhcQYEjdMdJVTUDfda2u): protocol: write tcp 188.166.225.170:40442->104.36.82.5:4864: use of closed network connection; write tcp 188.166.225.170:40442->104.36.82.5:4864: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40442->104.36.82.5:4864: use of closed network connection Aug 26 05:25:46.586886790 (13h22m13.745652525s) [seg_pos={0 0}, upload=11711]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:25:46.586944731 (13h22m13.745710465s) [upload=11711]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:25:46.586949191 (13h22m13.745714925s) [upload=11711]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:25:47.265618740 (13h22m14.424384474s) [upload=11711]: Done uploading segments... Aug 26 05:25:47.265645280 (13h22m14.424411014s) [upload=11711]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 58.722985494s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [..X...XX.........X...X...XXX.........X..X.....X.........XX.X] duration: 1.137750212s FAIL (2025-08-26 05:38:40 UTC) orbiter warning: operation: Commit uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 1.137774409s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 14 out of 60 is failed (threshold=3) [X..X...XX.........X...X...XXX.........X..X.....X.........XX.] duration: 23.118071154s FAIL (2025-08-26 05:38:28 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 23.118104538s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 6 out of 60 is failed (threshold=6) [.......X................XX...........X...X.................X] duration: 1.00483896s remote: "tcp:207.211.208.129:443" response headers: Accept-Ranges=bytes Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* X-Request-Id=DS54VnSBYuP Date=Tue, 26 Aug 2025 05:38:26 GMT Content-Type=text/html; charset=utf-8 target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:38:26 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 1.00486605s edge linksharing bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)