Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [..XXX.X.XXX..XX.......X.X.X...X.........XX..X...X.XX........] duration: 6.160031655s FAIL (2025-08-26 05:53:08 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 6.160063506s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [..XX..XX.XXXX...XXX..X....X.XX........XX.XX....X..XX........] duration: 2.178425499s FAIL (2025-08-26 05:53:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:53.712672629 (13h43m20.871438363s) [upload=12015]: Starting upload Aug 26 05:46:53.712770978 (13h43m20.871536712s) [upload=12015]: Done starting upload Aug 26 05:46:53.712826113 (13h43m20.871591847s) [upload=12015]: Uploading segments... Aug 26 05:46:53.712860629 (13h43m20.871626363s) [upload=12015]: Got next segment. Inline: true Aug 26 05:46:53.712864656 (13h43m20.87163039s) [upload=12015]: Waiting for error group managing segments... Aug 26 05:46:53.712910013 (13h43m20.871675748s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:53.712914472 (13h43m20.871680206s) [upload=12015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:53.712916816 (13h43m20.87168254s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:05.926670496 (13h43m33.08543623s) [upload=12015]: Done uploading segments... Aug 26 05:47:05.926688861 (13h43m33.085454595s) [upload=12015]: Upload finished. err: <nil> Duration:* 2.178454134s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [..X...XXX.........X..X.....X.........XX.XX.....XXXX.........] duration: 2.290274374s FAIL (2025-08-26 05:53:05 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 2.29030358s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [....XXX.X..XXX......X.X.XX.XX..XX.....X.....X....XXXX.......] duration: 2.606841433s FAIL (2025-08-26 05:53:02 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AF8B7429D78, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 2.606861892s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XXX.X..X...X....X..X.X.XXX.X...X...XXX.....XXX.XX...........] duration: 6.468098351s FAIL (2025-08-26 05:53:00 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AFA3987C431, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.468123961s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [........XX.X.....X.........X..X..........X......XXX.........] duration: 898.540561ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3AE5C0321C45 X-Request-Id=RvcWjVvwAah X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:45:38 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:52:59 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AE5C0321C45</RequestId><HostId></HostId></Error>" Duration: 898.568825ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [.....X..X..XXX..................X........XX.X...............] duration: 375.83774ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:52:58 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 375.878587ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X...X.X.....X.....X................X.XX.....X.X.........] duration: 6.110819141s FAIL (2025-08-26 05:52:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== upload Aug 26 05:46:16.882484883 (13h42m44.041250617s) [upload=12007]: Done starting upload Aug 26 05:46:16.882545439 (13h42m44.041311183s) [upload=12007]: Uploading segments... Aug 26 05:46:16.882768035 (13h42m44.041533769s) [upload=12007]: Got next segment. Inline: false Aug 26 05:46:16.882803943 (13h42m44.041569677s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:16.882809484 (13h42m44.041575238s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:21.138889236 (13h42m48.29765499s) [seg_pos={0 0}, upload=12007]: Begin upload segment... Aug 26 05:46:21.138982976 (13h42m48.29774871s) [seg_pos={0 0}, upload=12007]: Done begin upload segment. Aug 26 05:46:21.139040947 (13h42m48.297806681s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139512649 (13h42m48.298278413s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139625194 (13h42m48.298390928s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140172992 (13h42m48.298938726s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140281640 (13h42m48.299047374s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140707656 (13h42m48.29947339s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140761429 (13h42m48.299527163s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141248210 (13h42m48.300013944s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141308566 (13h42m48.3000743s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141759689 (13h42m48.300525423s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141823691 (13h42m48.300589445s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.142338075 (13h42m48.301103809s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143518584 (13h42m48.302284318s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143750468 (13h42m48.302516202s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145016009 (13h42m48.303781754s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145248584 (13h42m48.304014318s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145666915 (13h42m48.304432649s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.146108891 (13h42m48.304874625s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:22.291596419 (13h42m49.450362163s) [upload=12007]: Next returned nil segment Aug 26 05:46:22.291613341 (13h42m49.450379075s) [upload=12007]: Waiting for error group managing segments... Aug 26 05:46:22.579515464 (13h42m49.738281198s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.585387860 (13h42m49.744153594s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.983210064 (13h42m50.141975798s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.021005081 (13h42m50.179770815s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.037222411 (13h42m50.195988145s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.068168453 (13h42m50.226934187s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.087102512 (13h42m50.245868246s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.088458266 (13h42m50.247224s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.120248583 (13h42m50.279014317s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.124095745 (13h42m50.282861469s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.134317644 (13h42m50.293083378s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.172400171 (13h42m50.331165905s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.177768202 (13h42m50.336533936s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191352755 (13h42m50.350118489s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191378263 (13h42m50.350143998s) [seg_pos={0 0}, upload=12007]: Segment reached optimal threshold of 14 pieces. Aug 26 05:46:23.191493784 (13h42m50.350259518s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bkhuJ5T8SgGFMn6vv6wWnHiJarcMgVuhk6JNmNZoskzwP1gVi): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191553498 (13h42m50.350319232s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191602703 (13h42m50.350368437s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AKpkGZ6XbeqCTAYe273MACVvMCLcPPJ4By5EX2MkNuxC6zRpC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191626649 (13h42m50.350392383s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191677385 (13h42m50.350443119s) [seg_pos={0 0}, upload=12007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:46:23.191781304 (13h42m50.350547048s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:23.191789340 (13h42m50.350555104s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:29.417594986 (13h42m56.57636072s) [upload=12007]: Done uploading segments... Aug 26 05:46:29.417623151 (13h42m56.576388885s) [upload=12007]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 6.110841033s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [.....XX...X......X..........X........XX.XX.....XXX.X........] duration: 2.246177757s FAIL (2025-08-26 05:52:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:56.699225005 (13h43m23.857990739s) [upload=12016]: Starting upload Aug 26 05:46:56.699280191 (13h43m23.858045925s) [upload=12016]: Done starting upload Aug 26 05:46:56.699335787 (13h43m23.858101521s) [upload=12016]: Uploading segments... Aug 26 05:46:56.699375153 (13h43m23.858140887s) [upload=12016]: Got next segment. Inline: true Aug 26 05:46:56.699379361 (13h43m23.858145095s) [upload=12016]: Waiting for error group managing segments... Aug 26 05:46:56.699415379 (13h43m23.858181113s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:56.699419597 (13h43m23.858185331s) [upload=12016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:56.699422362 (13h43m23.858188097s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:59.085488757 (13h43m26.244254491s) [upload=12016]: Done uploading segments... Aug 26 05:46:59.085505859 (13h43m26.244271583s) [upload=12016]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 2.246222593s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [.........XX.X.....X.........X..X..........X......XXX........] duration: 848.638188ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3AE5C0321C45 X-Request-Id=RvcWjVvwAah X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:45:38 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:52:39 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AE5C0321C45</RequestId><HostId></HostId></Error>" Duration: 848.667485ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [......X..X..XXX..................X........XX.X..............] duration: 404.931708ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:52:39 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 404.95872ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.....X.XX........X........................X.....X...........] duration: 6.734159223s FAIL (2025-08-26 05:52:33 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3ADA6C640502, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.734180372s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [.......X..X..XXX..................X........XX.X.............] duration: 392.396713ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:52:22 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 392.417855ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 15 out of 60 is failed (threshold=3) [...X...XXX.........X..X.....X.........XX.XX.....XXXX........] duration: 2.238458167s FAIL (2025-08-26 05:52:17 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 2.238483505s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 7 out of 60 is failed (threshold=6) [.......XX...........X...X.................X.......X..X......] duration: 406.257426ms remote: "tcp:185.244.226.2:443" response headers: Date=Tue, 26 Aug 2025 05:46:59 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Accept-Ranges=bytes Access-Control-Allow-Origin=* X-Request-Id=TNpjirfo5py target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:52:17 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 406.287382ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [...XX..XX.XXXX...XXX..X....X.XX........XX.XX....X..XX.......] duration: 2.293350868s FAIL (2025-08-26 05:52:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:53.712672629 (13h43m20.871438363s) [upload=12015]: Starting upload Aug 26 05:46:53.712770978 (13h43m20.871536712s) [upload=12015]: Done starting upload Aug 26 05:46:53.712826113 (13h43m20.871591847s) [upload=12015]: Uploading segments... Aug 26 05:46:53.712860629 (13h43m20.871626363s) [upload=12015]: Got next segment. Inline: true Aug 26 05:46:53.712864656 (13h43m20.87163039s) [upload=12015]: Waiting for error group managing segments... Aug 26 05:46:53.712910013 (13h43m20.871675748s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:53.712914472 (13h43m20.871680206s) [upload=12015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:53.712916816 (13h43m20.87168254s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:05.926670496 (13h43m33.08543623s) [upload=12015]: Done uploading segments... Aug 26 05:47:05.926688861 (13h43m33.085454595s) [upload=12015]: Upload finished. err: <nil> Duration:* 2.293379273s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X...X.X.....X.....X................X.XX.....X.X........] duration: 6.135928877s FAIL (2025-08-26 05:52:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== upload Aug 26 05:46:16.882484883 (13h42m44.041250617s) [upload=12007]: Done starting upload Aug 26 05:46:16.882545439 (13h42m44.041311183s) [upload=12007]: Uploading segments... Aug 26 05:46:16.882768035 (13h42m44.041533769s) [upload=12007]: Got next segment. Inline: false Aug 26 05:46:16.882803943 (13h42m44.041569677s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:16.882809484 (13h42m44.041575238s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:21.138889236 (13h42m48.29765499s) [seg_pos={0 0}, upload=12007]: Begin upload segment... Aug 26 05:46:21.138982976 (13h42m48.29774871s) [seg_pos={0 0}, upload=12007]: Done begin upload segment. Aug 26 05:46:21.139040947 (13h42m48.297806681s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139512649 (13h42m48.298278413s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139625194 (13h42m48.298390928s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140172992 (13h42m48.298938726s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140281640 (13h42m48.299047374s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140707656 (13h42m48.29947339s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140761429 (13h42m48.299527163s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141248210 (13h42m48.300013944s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141308566 (13h42m48.3000743s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141759689 (13h42m48.300525423s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141823691 (13h42m48.300589445s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.142338075 (13h42m48.301103809s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143518584 (13h42m48.302284318s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143750468 (13h42m48.302516202s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145016009 (13h42m48.303781754s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145248584 (13h42m48.304014318s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145666915 (13h42m48.304432649s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.146108891 (13h42m48.304874625s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:22.291596419 (13h42m49.450362163s) [upload=12007]: Next returned nil segment Aug 26 05:46:22.291613341 (13h42m49.450379075s) [upload=12007]: Waiting for error group managing segments... Aug 26 05:46:22.579515464 (13h42m49.738281198s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.585387860 (13h42m49.744153594s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.983210064 (13h42m50.141975798s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.021005081 (13h42m50.179770815s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.037222411 (13h42m50.195988145s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.068168453 (13h42m50.226934187s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.087102512 (13h42m50.245868246s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.088458266 (13h42m50.247224s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.120248583 (13h42m50.279014317s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.124095745 (13h42m50.282861469s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.134317644 (13h42m50.293083378s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.172400171 (13h42m50.331165905s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.177768202 (13h42m50.336533936s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191352755 (13h42m50.350118489s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191378263 (13h42m50.350143998s) [seg_pos={0 0}, upload=12007]: Segment reached optimal threshold of 14 pieces. Aug 26 05:46:23.191493784 (13h42m50.350259518s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bkhuJ5T8SgGFMn6vv6wWnHiJarcMgVuhk6JNmNZoskzwP1gVi): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191553498 (13h42m50.350319232s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191602703 (13h42m50.350368437s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AKpkGZ6XbeqCTAYe273MACVvMCLcPPJ4By5EX2MkNuxC6zRpC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191626649 (13h42m50.350392383s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191677385 (13h42m50.350443119s) [seg_pos={0 0}, upload=12007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:46:23.191781304 (13h42m50.350547048s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:23.191789340 (13h42m50.350555104s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:29.417594986 (13h42m56.57636072s) [upload=12007]: Done uploading segments... Aug 26 05:46:29.417623151 (13h42m56.576388885s) [upload=12007]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 6.135955707s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [..X.X...XXX....XXX.......XXX.X..X....X....X.XXXX..X.XX......] duration: 6.069408204s FAIL (2025-08-26 05:52:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:43.718729348 (13h43m10.877495082s) [upload=12012]: Starting upload Aug 26 05:46:43.718782409 (13h43m10.877548143s) [upload=12012]: Done starting upload Aug 26 05:46:43.718860629 (13h43m10.877626363s) [upload=12012]: Uploading segments... Aug 26 05:46:43.719002550 (13h43m10.877768285s) [upload=12012]: Got next segment. Inline: false Aug 26 05:46:43.719035062 (13h43m10.877800797s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:43.719039982 (13h43m10.877805716s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:47.143071190 (13h43m14.301836925s) [seg_pos={0 0}, upload=12012]: Begin upload segment... Aug 26 05:46:47.143114072 (13h43m14.301879806s) [seg_pos={0 0}, upload=12012]: Done begin upload segment. Aug 26 05:46:47.143133920 (13h43m14.301899654s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143138138 (13h43m14.301903862s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143499169 (13h43m14.302264903s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143777411 (13h43m14.302543145s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144307807 (13h43m14.303073561s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144632718 (13h43m14.303398452s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144966477 (13h43m14.303732201s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145197318 (13h43m14.303963042s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145436065 (13h43m14.304201799s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145716882 (13h43m14.304482616s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145976660 (13h43m14.304742384s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146192533 (13h43m14.304958267s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146405400 (13h43m14.305171134s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146596605 (13h43m14.305362339s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146840753 (13h43m14.305606487s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.147135667 (13h43m14.305901401s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:48.890361729 (13h43m16.049127464s) [upload=12012]: Next returned nil segment Aug 26 05:46:48.890387599 (13h43m16.049153333s) [upload=12012]: Waiting for error group managing segments... Aug 26 05:46:49.173838475 (13h43m16.332604209s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.180370984 (13h43m16.339136718s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.185449833 (13h43m16.344215567s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.190831099 (13h43m16.349596833s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.198390243 (13h43m16.357155977s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.216665500 (13h43m16.375431234s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.219857638 (13h43m16.378623372s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.229492494 (13h43m16.388258228s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.233073916 (13h43m16.39183965s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.234706269 (13h43m16.393472004s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.237854213 (13h43m16.396619947s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.245653225 (13h43m16.404418959s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251526182 (13h43m16.410291916s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251541232 (13h43m16.410306966s) [seg_pos={0 0}, upload=12012]: Segment reached optimal threshold of 13 pieces. Aug 26 05:46:49.251641053 (13h43m16.410406787s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251687080 (13h43m16.410452814s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251697882 (13h43m16.410463616s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251789296 (13h43m16.41055503s) [seg_pos={0 0}, upload=12012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:46:49.251862136 (13h43m16.41062787s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:49.251868428 (13h43m16.410634162s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:13.300278094 (13h43m40.459043828s) [upload=12012]: Done uploading segments... Aug 26 05:47:13.300325886 (13h43m40.45909162s) [upload=12012]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.069435928s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [...XXX.X.XXX..XX.......X.X.X...X.........XX..X...X.XX.......] duration: 5.934639818s FAIL (2025-08-26 05:52:10 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.934668382s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [......XX...X......X..........X........XX.XX.....XXX.X.......] duration: 2.316710834s FAIL (2025-08-26 05:52:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:56.699225005 (13h43m23.857990739s) [upload=12016]: Starting upload Aug 26 05:46:56.699280191 (13h43m23.858045925s) [upload=12016]: Done starting upload Aug 26 05:46:56.699335787 (13h43m23.858101521s) [upload=12016]: Uploading segments... Aug 26 05:46:56.699375153 (13h43m23.858140887s) [upload=12016]: Got next segment. Inline: true Aug 26 05:46:56.699379361 (13h43m23.858145095s) [upload=12016]: Waiting for error group managing segments... Aug 26 05:46:56.699415379 (13h43m23.858181113s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:56.699419597 (13h43m23.858185331s) [upload=12016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:56.699422362 (13h43m23.858188097s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:59.085488757 (13h43m26.244254491s) [upload=12016]: Done uploading segments... Aug 26 05:46:59.085505859 (13h43m26.244271583s) [upload=12016]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 2.316740853s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [..........XX.X.....X.........X..X..........X......XXX.......] duration: 896.672747ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3AE5C0321C45 X-Request-Id=RvcWjVvwAah X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:45:38 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:52:04 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AE5C0321C45</RequestId><HostId></HostId></Error>" Duration: 896.725276ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [XXXX.X..X...X....X..X.X.XXX.X...X...XXX.....XXX.XX..........] duration: 6.469403678s FAIL (2025-08-26 05:52:04 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AFA3987C431, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.469429797s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 16 out of 60 is failed (threshold=3) [X...X...XXX.........X..X.....X.........XX.XX.....XXXX.......] duration: 2.333758626s FAIL (2025-08-26 05:52:01 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 2.333793442s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [.....XXX.X..XXX......X.X.XX.XX..XX.....X.....X....XXXX......] duration: 2.802619906s FAIL (2025-08-26 05:51:55 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AF8B7429D78, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 2.802644173s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [....XX..XX.XXXX...XXX..X....X.XX........XX.XX....X..XX......] duration: 2.593474167s FAIL (2025-08-26 05:51:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:53.712672629 (13h43m20.871438363s) [upload=12015]: Starting upload Aug 26 05:46:53.712770978 (13h43m20.871536712s) [upload=12015]: Done starting upload Aug 26 05:46:53.712826113 (13h43m20.871591847s) [upload=12015]: Uploading segments... Aug 26 05:46:53.712860629 (13h43m20.871626363s) [upload=12015]: Got next segment. Inline: true Aug 26 05:46:53.712864656 (13h43m20.87163039s) [upload=12015]: Waiting for error group managing segments... Aug 26 05:46:53.712910013 (13h43m20.871675748s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:53.712914472 (13h43m20.871680206s) [upload=12015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:53.712916816 (13h43m20.87168254s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:05.926670496 (13h43m33.08543623s) [upload=12015]: Done uploading segments... Aug 26 05:47:05.926688861 (13h43m33.085454595s) [upload=12015]: Upload finished. err: <nil> Duration:* 2.593508123s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [...X.X...XXX....XXX.......XXX.X..X....X....X.XXXX..X.XX.....] duration: 6.074180226s FAIL (2025-08-26 05:51:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:43.718729348 (13h43m10.877495082s) [upload=12012]: Starting upload Aug 26 05:46:43.718782409 (13h43m10.877548143s) [upload=12012]: Done starting upload Aug 26 05:46:43.718860629 (13h43m10.877626363s) [upload=12012]: Uploading segments... Aug 26 05:46:43.719002550 (13h43m10.877768285s) [upload=12012]: Got next segment. Inline: false Aug 26 05:46:43.719035062 (13h43m10.877800797s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:43.719039982 (13h43m10.877805716s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:47.143071190 (13h43m14.301836925s) [seg_pos={0 0}, upload=12012]: Begin upload segment... Aug 26 05:46:47.143114072 (13h43m14.301879806s) [seg_pos={0 0}, upload=12012]: Done begin upload segment. Aug 26 05:46:47.143133920 (13h43m14.301899654s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143138138 (13h43m14.301903862s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143499169 (13h43m14.302264903s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143777411 (13h43m14.302543145s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144307807 (13h43m14.303073561s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144632718 (13h43m14.303398452s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144966477 (13h43m14.303732201s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145197318 (13h43m14.303963042s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145436065 (13h43m14.304201799s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145716882 (13h43m14.304482616s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145976660 (13h43m14.304742384s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146192533 (13h43m14.304958267s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146405400 (13h43m14.305171134s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146596605 (13h43m14.305362339s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146840753 (13h43m14.305606487s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.147135667 (13h43m14.305901401s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:48.890361729 (13h43m16.049127464s) [upload=12012]: Next returned nil segment Aug 26 05:46:48.890387599 (13h43m16.049153333s) [upload=12012]: Waiting for error group managing segments... Aug 26 05:46:49.173838475 (13h43m16.332604209s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.180370984 (13h43m16.339136718s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.185449833 (13h43m16.344215567s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.190831099 (13h43m16.349596833s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.198390243 (13h43m16.357155977s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.216665500 (13h43m16.375431234s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.219857638 (13h43m16.378623372s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.229492494 (13h43m16.388258228s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.233073916 (13h43m16.39183965s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.234706269 (13h43m16.393472004s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.237854213 (13h43m16.396619947s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.245653225 (13h43m16.404418959s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251526182 (13h43m16.410291916s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251541232 (13h43m16.410306966s) [seg_pos={0 0}, upload=12012]: Segment reached optimal threshold of 13 pieces. Aug 26 05:46:49.251641053 (13h43m16.410406787s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251687080 (13h43m16.410452814s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251697882 (13h43m16.410463616s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251789296 (13h43m16.41055503s) [seg_pos={0 0}, upload=12012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:46:49.251862136 (13h43m16.41062787s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:49.251868428 (13h43m16.410634162s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:13.300278094 (13h43m40.459043828s) [upload=12012]: Done uploading segments... Aug 26 05:47:13.300325886 (13h43m40.45909162s) [upload=12012]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.074206827s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......X.XX........X........................X.....X..........] duration: 6.64615801s FAIL (2025-08-26 05:51:23 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3ADA6C640502, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.646197938s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [........X..X..XXX..................X........XX.X............] duration: 378.688294ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:51:22 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 378.714926ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 7 out of 60 is failed (threshold=6) [........XX...........X...X.................X.......X..X.....] duration: 501.736965ms remote: "tcp:185.244.226.2:443" response headers: Date=Tue, 26 Aug 2025 05:46:59 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Accept-Ranges=bytes Access-Control-Allow-Origin=* X-Request-Id=TNpjirfo5py target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:51:21 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 501.761392ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [....XXX.X.XXX..XX.......X.X.X...X.........XX..X...X.XX......] duration: 5.896217722s FAIL (2025-08-26 05:51:20 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 5.8962484s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X...X.X.....X.....X................X.XX.....X.X.......] duration: 6.010378618s FAIL (2025-08-26 05:51:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== upload Aug 26 05:46:16.882484883 (13h42m44.041250617s) [upload=12007]: Done starting upload Aug 26 05:46:16.882545439 (13h42m44.041311183s) [upload=12007]: Uploading segments... Aug 26 05:46:16.882768035 (13h42m44.041533769s) [upload=12007]: Got next segment. Inline: false Aug 26 05:46:16.882803943 (13h42m44.041569677s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:16.882809484 (13h42m44.041575238s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:21.138889236 (13h42m48.29765499s) [seg_pos={0 0}, upload=12007]: Begin upload segment... Aug 26 05:46:21.138982976 (13h42m48.29774871s) [seg_pos={0 0}, upload=12007]: Done begin upload segment. Aug 26 05:46:21.139040947 (13h42m48.297806681s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139512649 (13h42m48.298278413s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139625194 (13h42m48.298390928s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140172992 (13h42m48.298938726s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140281640 (13h42m48.299047374s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140707656 (13h42m48.29947339s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140761429 (13h42m48.299527163s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141248210 (13h42m48.300013944s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141308566 (13h42m48.3000743s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141759689 (13h42m48.300525423s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141823691 (13h42m48.300589445s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.142338075 (13h42m48.301103809s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143518584 (13h42m48.302284318s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143750468 (13h42m48.302516202s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145016009 (13h42m48.303781754s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145248584 (13h42m48.304014318s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145666915 (13h42m48.304432649s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.146108891 (13h42m48.304874625s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:22.291596419 (13h42m49.450362163s) [upload=12007]: Next returned nil segment Aug 26 05:46:22.291613341 (13h42m49.450379075s) [upload=12007]: Waiting for error group managing segments... Aug 26 05:46:22.579515464 (13h42m49.738281198s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.585387860 (13h42m49.744153594s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.983210064 (13h42m50.141975798s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.021005081 (13h42m50.179770815s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.037222411 (13h42m50.195988145s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.068168453 (13h42m50.226934187s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.087102512 (13h42m50.245868246s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.088458266 (13h42m50.247224s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.120248583 (13h42m50.279014317s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.124095745 (13h42m50.282861469s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.134317644 (13h42m50.293083378s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.172400171 (13h42m50.331165905s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.177768202 (13h42m50.336533936s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191352755 (13h42m50.350118489s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191378263 (13h42m50.350143998s) [seg_pos={0 0}, upload=12007]: Segment reached optimal threshold of 14 pieces. Aug 26 05:46:23.191493784 (13h42m50.350259518s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bkhuJ5T8SgGFMn6vv6wWnHiJarcMgVuhk6JNmNZoskzwP1gVi): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191553498 (13h42m50.350319232s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191602703 (13h42m50.350368437s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AKpkGZ6XbeqCTAYe273MACVvMCLcPPJ4By5EX2MkNuxC6zRpC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191626649 (13h42m50.350392383s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191677385 (13h42m50.350443119s) [seg_pos={0 0}, upload=12007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:46:23.191781304 (13h42m50.350547048s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:23.191789340 (13h42m50.350555104s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:29.417594986 (13h42m56.57636072s) [upload=12007]: Done uploading segments... Aug 26 05:46:29.417623151 (13h42m56.576388885s) [upload=12007]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 6.010414036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [.......XX...X......X..........X........XX.XX.....XXX.X......] duration: 2.175324356s FAIL (2025-08-26 05:51:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:56.699225005 (13h43m23.857990739s) [upload=12016]: Starting upload Aug 26 05:46:56.699280191 (13h43m23.858045925s) [upload=12016]: Done starting upload Aug 26 05:46:56.699335787 (13h43m23.858101521s) [upload=12016]: Uploading segments... Aug 26 05:46:56.699375153 (13h43m23.858140887s) [upload=12016]: Got next segment. Inline: true Aug 26 05:46:56.699379361 (13h43m23.858145095s) [upload=12016]: Waiting for error group managing segments... Aug 26 05:46:56.699415379 (13h43m23.858181113s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:56.699419597 (13h43m23.858185331s) [upload=12016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:56.699422362 (13h43m23.858188097s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:59.085488757 (13h43m26.244254491s) [upload=12016]: Done uploading segments... Aug 26 05:46:59.085505859 (13h43m26.244271583s) [upload=12016]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 2.175347611s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [...........XX.X.....X.........X..X..........X......XXX......] duration: 846.409663ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3AE5C0321C45 X-Request-Id=RvcWjVvwAah X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:45:38 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:51:10 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AE5C0321C45</RequestId><HostId></HostId></Error>" Duration: 846.436997ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 16 out of 60 is failed (threshold=3) [.X...X...XXX.........X..X.....X.........XX.XX.....XXXX......] duration: 2.917296069s FAIL (2025-08-26 05:50:55 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 2.91732258s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.XXXX.X..X...X....X..X.X.XXX.X...X...XXX.....XXX.XX.........] duration: 6.223426799s FAIL (2025-08-26 05:50:53 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AFA3987C431, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.223451816s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 21 out of 60 is failed (threshold=3) [......XXX.X..XXX......X.X.XX.XX..XX.....X.....X....XXXX.....] duration: 2.699670093s FAIL (2025-08-26 05:50:53 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AF8B7429D78, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 2.699692865s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [.....XXX.X.XXX..XX.......X.X.X...X.........XX..X...X.XX.....] duration: 6.24739682s FAIL (2025-08-26 05:50:48 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 6.247451254s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 22 out of 60 is failed (threshold=3) [.....XX..XX.XXXX...XXX..X....X.XX........XX.XX....X..XX.....] duration: 2.135574378s FAIL (2025-08-26 05:50:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:53.712672629 (13h43m20.871438363s) [upload=12015]: Starting upload Aug 26 05:46:53.712770978 (13h43m20.871536712s) [upload=12015]: Done starting upload Aug 26 05:46:53.712826113 (13h43m20.871591847s) [upload=12015]: Uploading segments... Aug 26 05:46:53.712860629 (13h43m20.871626363s) [upload=12015]: Got next segment. Inline: true Aug 26 05:46:53.712864656 (13h43m20.87163039s) [upload=12015]: Waiting for error group managing segments... Aug 26 05:46:53.712910013 (13h43m20.871675748s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:53.712914472 (13h43m20.871680206s) [upload=12015]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:53.712916816 (13h43m20.87168254s) [upload=12015]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:05.926670496 (13h43m33.08543623s) [upload=12015]: Done uploading segments... Aug 26 05:47:05.926688861 (13h43m33.085454595s) [upload=12015]: Upload finished. err: <nil> Duration:* 2.1356019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X...X.X.....X.....X................X.XX.....X.X......] duration: 5.875782491s FAIL (2025-08-26 05:50:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== upload Aug 26 05:46:16.882484883 (13h42m44.041250617s) [upload=12007]: Done starting upload Aug 26 05:46:16.882545439 (13h42m44.041311183s) [upload=12007]: Uploading segments... Aug 26 05:46:16.882768035 (13h42m44.041533769s) [upload=12007]: Got next segment. Inline: false Aug 26 05:46:16.882803943 (13h42m44.041569677s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:16.882809484 (13h42m44.041575238s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:21.138889236 (13h42m48.29765499s) [seg_pos={0 0}, upload=12007]: Begin upload segment... Aug 26 05:46:21.138982976 (13h42m48.29774871s) [seg_pos={0 0}, upload=12007]: Done begin upload segment. Aug 26 05:46:21.139040947 (13h42m48.297806681s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139512649 (13h42m48.298278413s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139625194 (13h42m48.298390928s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140172992 (13h42m48.298938726s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140281640 (13h42m48.299047374s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140707656 (13h42m48.29947339s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140761429 (13h42m48.299527163s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141248210 (13h42m48.300013944s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141308566 (13h42m48.3000743s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141759689 (13h42m48.300525423s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141823691 (13h42m48.300589445s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.142338075 (13h42m48.301103809s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143518584 (13h42m48.302284318s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143750468 (13h42m48.302516202s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145016009 (13h42m48.303781754s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145248584 (13h42m48.304014318s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145666915 (13h42m48.304432649s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.146108891 (13h42m48.304874625s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:22.291596419 (13h42m49.450362163s) [upload=12007]: Next returned nil segment Aug 26 05:46:22.291613341 (13h42m49.450379075s) [upload=12007]: Waiting for error group managing segments... Aug 26 05:46:22.579515464 (13h42m49.738281198s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.585387860 (13h42m49.744153594s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.983210064 (13h42m50.141975798s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.021005081 (13h42m50.179770815s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.037222411 (13h42m50.195988145s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.068168453 (13h42m50.226934187s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.087102512 (13h42m50.245868246s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.088458266 (13h42m50.247224s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.120248583 (13h42m50.279014317s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.124095745 (13h42m50.282861469s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.134317644 (13h42m50.293083378s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.172400171 (13h42m50.331165905s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.177768202 (13h42m50.336533936s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191352755 (13h42m50.350118489s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191378263 (13h42m50.350143998s) [seg_pos={0 0}, upload=12007]: Segment reached optimal threshold of 14 pieces. Aug 26 05:46:23.191493784 (13h42m50.350259518s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bkhuJ5T8SgGFMn6vv6wWnHiJarcMgVuhk6JNmNZoskzwP1gVi): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191553498 (13h42m50.350319232s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191602703 (13h42m50.350368437s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AKpkGZ6XbeqCTAYe273MACVvMCLcPPJ4By5EX2MkNuxC6zRpC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191626649 (13h42m50.350392383s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191677385 (13h42m50.350443119s) [seg_pos={0 0}, upload=12007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:46:23.191781304 (13h42m50.350547048s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:23.191789340 (13h42m50.350555104s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:29.417594986 (13h42m56.57636072s) [upload=12007]: Done uploading segments... Aug 26 05:46:29.417623151 (13h42m56.576388885s) [upload=12007]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 5.875806266s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.......X.XX........X........................X.....X.........] duration: 6.948266208s FAIL (2025-08-26 05:50:40 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3ADA6C640502, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.948301997s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [..XXXX.X..X...X....X..X.X.XXX.X...X...XXX.....XXX.XX........] duration: 6.09632563s FAIL (2025-08-26 05:50:37 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AFA3987C431, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.09634157s edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 7 out of 60 is failed (threshold=6) [.........XX...........X...X.................X.......X..X....] duration: 475.7797ms remote: "tcp:185.244.226.2:443" response headers: Date=Tue, 26 Aug 2025 05:46:59 GMT Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* Access-Control-Allow-Methods=GET, HEAD Accept-Ranges=bytes Access-Control-Allow-Origin=* X-Request-Id=TNpjirfo5py target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 05:50:34 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 475.808904ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [....X.X...XXX....XXX.......XXX.X..X....X....X.XXXX..X.XX....] duration: 6.283338337s FAIL (2025-08-26 05:50:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 05:46:43.718729348 (13h43m10.877495082s) [upload=12012]: Starting upload Aug 26 05:46:43.718782409 (13h43m10.877548143s) [upload=12012]: Done starting upload Aug 26 05:46:43.718860629 (13h43m10.877626363s) [upload=12012]: Uploading segments... Aug 26 05:46:43.719002550 (13h43m10.877768285s) [upload=12012]: Got next segment. Inline: false Aug 26 05:46:43.719035062 (13h43m10.877800797s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:43.719039982 (13h43m10.877805716s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:47.143071190 (13h43m14.301836925s) [seg_pos={0 0}, upload=12012]: Begin upload segment... Aug 26 05:46:47.143114072 (13h43m14.301879806s) [seg_pos={0 0}, upload=12012]: Done begin upload segment. Aug 26 05:46:47.143133920 (13h43m14.301899654s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143138138 (13h43m14.301903862s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143499169 (13h43m14.302264903s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.143777411 (13h43m14.302543145s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144307807 (13h43m14.303073561s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144632718 (13h43m14.303398452s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.144966477 (13h43m14.303732201s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145197318 (13h43m14.303963042s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145436065 (13h43m14.304201799s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145716882 (13h43m14.304482616s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.145976660 (13h43m14.304742384s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146192533 (13h43m14.304958267s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146405400 (13h43m14.305171134s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146596605 (13h43m14.305362339s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.146840753 (13h43m14.305606487s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:47.147135667 (13h43m14.305901401s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Uploading piece... Aug 26 05:46:48.890361729 (13h43m16.049127464s) [upload=12012]: Next returned nil segment Aug 26 05:46:48.890387599 (13h43m16.049153333s) [upload=12012]: Waiting for error group managing segments... Aug 26 05:46:49.173838475 (13h43m16.332604209s) [noise=true , address=45.91.116.8:4047 , piece_id=6OFEFEQ4CPJNWF6RFWXVI453XXSCLWKUQK2SXOBVT5EN5W7Q2PGQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.180370984 (13h43m16.339136718s) [noise=true , address=77.90.31.16:4734 , piece_id=J23BNYNFYIXTFBBREUKW43LQX43K2E5YGP4GYT6QYNCJLFKH7CKQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.185449833 (13h43m16.344215567s) [noise=true , address=45.91.116.6:4975 , piece_id=5W2C352E6WUD7TC7LEK62GOJPA244ETVU3DKMW73CSB7I5PFTQ5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.190831099 (13h43m16.349596833s) [noise=true , address=45.91.116.6:4973 , piece_id=JTHRCHK3P4JXR2ZYSRLRUJTOHLCNSACM3JUZB5H77YXGKWJDVC6Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.198390243 (13h43m16.357155977s) [noise=true , address=104.36.82.4:4801 , piece_id=RZDP5CICQ6675YLV3GXBAM3AH5RUQPWZENWHOVXKQAB3PLW3WDTA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.216665500 (13h43m16.375431234s) [noise=true , address=77.90.31.11:4554 , piece_id=ZDHZSYLMR4W7GSJP7IUFEIDT3HSTWF2MS2OZTRT7EER3LBWVWX4A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.219857638 (13h43m16.378623372s) [noise=true , address=77.90.31.15:4697 , piece_id=ZAC4NE5DFXQMFKTPVTE2EKZC6TQ56SGG7G2TES36S5WS6EBPT3SQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.229492494 (13h43m16.388258228s) [noise=true , address=77.90.31.14:4663 , piece_id=TPQS7EUXQ5B625IAAIMRGSIDS65DSMKATLVQRYEWWZFJG2MCB7KQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.233073916 (13h43m16.39183965s) [noise=true , address=104.36.82.5:4838 , piece_id=ZEK6RZDMBU6AOYMULGB37GNSN2LWUAH3KRMETQZOKCAPCPYZ4AEA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.234706269 (13h43m16.393472004s) [noise=true , address=77.90.31.15:4699 , piece_id=A7LTWXC7ORFMZ24PPHEO7R2DV4SEJ46I5YAXIKLM7FGRMW5TMJYQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.237854213 (13h43m16.396619947s) [noise=true , address=77.90.31.18:4805 , piece_id=B2NKHNGWJAQXEDDTNTLSHUVK6Y4ZO6EK6F2TKG5WVLVCAI5SYEOQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.245653225 (13h43m16.404418959s) [noise=true , address=104.36.82.10:4079 , piece_id=LZG5OLWCUMLVU7NRVOQ5UJEMX5CTPIS3Z3ZSEW2JBOJA2GXVWP3Q, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251526182 (13h43m16.410291916s) [noise=true , address=45.91.116.10:4082 , piece_id=2E5TBPQXOHHMRCWRXG3YG3CM35D7BV3BTDPL2JNACQYO6A2OZTLQ, seg_pos={0 0}, upload=12012]: Done uploading piece. err: <nil> Aug 26 05:46:49.251541232 (13h43m16.410306966s) [seg_pos={0 0}, upload=12012]: Segment reached optimal threshold of 13 pieces. Aug 26 05:46:49.251641053 (13h43m16.410406787s) [noise=true , address=45.91.116.8:4048 , piece_id=NMFQJE2YTCCKQP2LVFIKB3P6ZCAX7KPOCPWRARRLRAGDCWWR7UBA, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251687080 (13h43m16.410452814s) [noise=true , address=104.36.82.6:4876 , piece_id=UCBIXJBMQR7MHYX66N3XJVU72JRFA6NGLC35XDLPPH3IJOYMDL5A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251697882 (13h43m16.410463616s) [noise=true , address=45.91.116.11:4119 , piece_id=X3E6TMHYHQML64HGW5E4S6JMZ45XO72H7GYHQ34NR5WSDV725Q2A, seg_pos={0 0}, upload=12012]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:49.251789296 (13h43m16.41055503s) [seg_pos={0 0}, upload=12012]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 05:46:49.251862136 (13h43m16.41062787s) [upload=12012]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:49.251868428 (13h43m16.410634162s) [upload=12012]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:47:13.300278094 (13h43m40.459043828s) [upload=12012]: Done uploading segments... Aug 26 05:47:13.300325886 (13h43m40.45909162s) [upload=12012]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.283368384s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 13 out of 60 is failed (threshold=3) [........XX...X......X..........X........XX.XX.....XXX.X.....] duration: 2.274007487s FAIL (2025-08-26 05:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 05:46:56.699225005 (13h43m23.857990739s) [upload=12016]: Starting upload Aug 26 05:46:56.699280191 (13h43m23.858045925s) [upload=12016]: Done starting upload Aug 26 05:46:56.699335787 (13h43m23.858101521s) [upload=12016]: Uploading segments... Aug 26 05:46:56.699375153 (13h43m23.858140887s) [upload=12016]: Got next segment. Inline: true Aug 26 05:46:56.699379361 (13h43m23.858145095s) [upload=12016]: Waiting for error group managing segments... Aug 26 05:46:56.699415379 (13h43m23.858181113s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:56.699419597 (13h43m23.858185331s) [upload=12016]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 05:46:56.699422362 (13h43m23.858188097s) [upload=12016]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:59.085488757 (13h43m26.244254491s) [upload=12016]: Done uploading segments... Aug 26 05:46:59.085505859 (13h43m26.244271583s) [upload=12016]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 2.274036272s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 10 out of 60 is failed (threshold=3) [............XX.X.....X.........X..X..........X......XXX.....] duration: 807.954424ms remote: "tcp:45.38.24.2:443" response headers: Accept-Ranges=bytes Content-Length=433 Content-Type=application/xml X-Amz-Request-Id=185F3AE5C0321C45 X-Request-Id=RvcWjVvwAah X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:45:38 GMT Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 05:50:19 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AE5C0321C45</RequestId><HostId></HostId></Error>" Duration: 807.978951ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 9 out of 60 is failed (threshold=3) [.........X..X..XXX..................X........XX.X...........] duration: 449.415534ms remote: "tcp:207.211.208.129:443" response headers: Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=Rt9Df2xLHNU X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=185F3A9D950F4F3D Date=Tue, 26 Aug 2025 05:40:28 GMT Content-Length=423 target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 05:50:17 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3A9D950F4F3D</RequestId><HostId></HostId></Error>" Duration: 449.447315ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [...XXXX.X..X...X....X..X.X.XXX.X...X...XXX.....XXX.XX.......] duration: 6.463234455s FAIL (2025-08-26 05:50:17 UTC) orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AFA3987C431, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.463302204s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X...X.X.....X.....X................X.XX.....X.X.....] duration: 6.129350418s FAIL (2025-08-26 05:50:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== upload Aug 26 05:46:16.882484883 (13h42m44.041250617s) [upload=12007]: Done starting upload Aug 26 05:46:16.882545439 (13h42m44.041311183s) [upload=12007]: Uploading segments... Aug 26 05:46:16.882768035 (13h42m44.041533769s) [upload=12007]: Got next segment. Inline: false Aug 26 05:46:16.882803943 (13h42m44.041569677s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:46:16.882809484 (13h42m44.041575238s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:46:21.138889236 (13h42m48.29765499s) [seg_pos={0 0}, upload=12007]: Begin upload segment... Aug 26 05:46:21.138982976 (13h42m48.29774871s) [seg_pos={0 0}, upload=12007]: Done begin upload segment. Aug 26 05:46:21.139040947 (13h42m48.297806681s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139512649 (13h42m48.298278413s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.139625194 (13h42m48.298390928s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140172992 (13h42m48.298938726s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140281640 (13h42m48.299047374s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140707656 (13h42m48.29947339s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.140761429 (13h42m48.299527163s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141248210 (13h42m48.300013944s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141308566 (13h42m48.3000743s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141759689 (13h42m48.300525423s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.141823691 (13h42m48.300589445s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.142338075 (13h42m48.301103809s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143518584 (13h42m48.302284318s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.143750468 (13h42m48.302516202s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145016009 (13h42m48.303781754s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145248584 (13h42m48.304014318s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.145666915 (13h42m48.304432649s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:21.146108891 (13h42m48.304874625s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Uploading piece... Aug 26 05:46:22.291596419 (13h42m49.450362163s) [upload=12007]: Next returned nil segment Aug 26 05:46:22.291613341 (13h42m49.450379075s) [upload=12007]: Waiting for error group managing segments... Aug 26 05:46:22.579515464 (13h42m49.738281198s) [noise=true , address=79.127.205.234:4619 , piece_id=SIHRJBFDTLFYPQZOGJLVL35E3OJDEY6IRG353WQ75HCAQY27B75Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.585387860 (13h42m49.744153594s) [noise=true , address=79.127.205.246:4277 , piece_id=XAL7JR4I44VI25IRMZY2B5KIXWHML3E3NSBNF7ISDXFVLYLLWBZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:22.983210064 (13h42m50.141975798s) [noise=true , address=79.127.220.98:4411 , piece_id=DARBAJI7CVPXIF2QWRJLAZI7MZWF4AKTD6MCXKKDGUOCN4MJKHEQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.021005081 (13h42m50.179770815s) [noise=true , address=45.91.116.10:4106 , piece_id=LTSNKOLAFBKGC5LV7YEITOO4RMZJKFEHJGMGH3XTIIOYU4IPKH4Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.037222411 (13h42m50.195988145s) [noise=true , address=45.91.116.12:4254 , piece_id=AU5PRPXNCGJPP3YET6L2VGTHNP6YAOM47VGQOBSJLNLMM3FPTHNA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.068168453 (13h42m50.226934187s) [noise=true , address=104.36.82.44:4891 , piece_id=LQMXGJDVIIAHWKXV4UZ4WQV37SQOJFUBN4BBPO4UL6OLEZXNRBUQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.087102512 (13h42m50.245868246s) [noise=true , address=45.91.116.7:4030 , piece_id=MPWVSA23WSGP76IWZKVE7SVSONKUEUZP5GVUY7L26UC5BP4WNSHA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.088458266 (13h42m50.247224s) [noise=true , address=104.36.82.60:4341 , piece_id=AVKZXP2XTMUE5IZPFYFZWA4IW2UHMY2AVBWVXSLEILQFIDDE2Z2Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.120248583 (13h42m50.279014317s) [noise=true , address=79.127.201.215:4351 , piece_id=FLLBZAHOZYQBNINMOCJIEHNQPG6KBDDDRJHUUSN2WP7NAW2K7IGA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.124095745 (13h42m50.282861469s) [noise=true , address=79.127.201.215:4349 , piece_id=E3DJJKTXPY6UHIZF4DERGYBXNYBRJ3MTN4ANMTHHHLLWASJ5642Q, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.134317644 (13h42m50.293083378s) [noise=true , address=104.36.82.11:4127 , piece_id=BCFA3LJSZHLSRSXYURM7ISSYMO3J4BTPT3HQ62FGG6KROV4TANTQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.172400171 (13h42m50.331165905s) [noise=true , address=45.91.116.14:4201 , piece_id=L7E4D5O4CGDVSLDXLEFM5K6N3DUGIDEZIROQK5TBYW6OA3O5QNLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.177768202 (13h42m50.336533936s) [noise=true , address=77.90.31.12:4600 , piece_id=RTQ2BL2WGYWTFF3EVFWN4PXZYYX3PRGDDSRW36A5ONJSLJE2ZWAA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191352755 (13h42m50.350118489s) [noise=true , address=104.36.82.44:4892 , piece_id=MTUN2XAFG3BRUST4SSHIKZWENWX2XAIPA6GBBRV23ILNMP4ZB7JA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: <nil> Aug 26 05:46:23.191378263 (13h42m50.350143998s) [seg_pos={0 0}, upload=12007]: Segment reached optimal threshold of 14 pieces. Aug 26 05:46:23.191493784 (13h42m50.350259518s) [noise=true , address=77.90.31.11:4561 , piece_id=M4TKHRGO4JZECAS7W4KLJSSWZO36Z7KCH2NGGYIBQONG752PQXZA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bkhuJ5T8SgGFMn6vv6wWnHiJarcMgVuhk6JNmNZoskzwP1gVi): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191553498 (13h42m50.350319232s) [noise=true , address=104.36.82.50:4044 , piece_id=MOZSB7K2XMD5CCBYK5EBT4RSJ523EKT2VPX2KQDDIQDDBN7AVRLA, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oEaCgnsStJePrKHSXLLmwsAuYPEmgFbejFAMmNMqLp8m8vxiC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191602703 (13h42m50.350368437s) [noise=true , address=104.36.82.23:4473 , piece_id=FG7MXSUBVVCBFCPKMXAFETXFCDZEOFHS7JNDSL3RJZARM3TMVBYQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1AKpkGZ6XbeqCTAYe273MACVvMCLcPPJ4By5EX2MkNuxC6zRpC): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191626649 (13h42m50.350392383s) [noise=true , address=104.36.82.13:4204 , piece_id=3RWAGONF3VCOEMHWO762C5H66U2WIGGDNLYYWLZGXFN26GAXULWQ, seg_pos={0 0}, upload=12007]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12R53RRLYuJc7r9Z1W9n6guzrr5wF4T4vuGMEzjWAPBJwSCJq1g): protocol: expected piece hash; context canceled; EOF Aug 26 05:46:23.191677385 (13h42m50.350443119s) [seg_pos={0 0}, upload=12007]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:46:23.191781304 (13h42m50.350547048s) [upload=12007]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:46:23.191789340 (13h42m50.350555104s) [upload=12007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:46:29.417594986 (13h42m56.57636072s) [upload=12007]: Done uploading segments... Aug 26 05:46:29.417623151 (13h42m56.576388885s) [upload=12007]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 6.12938235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [........X.XX........X........................X.....X........] duration: 6.679002506s FAIL (2025-08-26 05:50:10 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3ADA6C640502, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 6.679031651s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 19 out of 60 is failed (threshold=3) [......XXX.X.XXX..XX.......X.X.X...X.........XX..X...X.XX....] duration: 7.793926014s FAIL (2025-08-26 05:50:01 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 7.793951524s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)