Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [...X............................................XX..........] duration:
3.443499246s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:40:03 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.443530204s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.297356735s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:39:58 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.297400187s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
283.084576ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:39:54 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 283.125153ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
252.863271ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:39:40 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 252.894471ms
edge
gateway
vhost
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.105315239s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:39:31 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.105343494s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
25.406145ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:39:27 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 25.436865ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [........X................................................X..] duration:
483.545287ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:39:25 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 483.570945ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
259.323873ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:39:18 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 259.354521ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.782189353s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:39:16 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.782237605s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
317.814515ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:39:14 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 317.841156ms
edge
linksharing
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [......X.XX.....X......................................X.....] duration:
5.671610698s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:39:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.671640455s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [................................................X...........] duration:
266.36032ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:39:09 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 266.384507ms
edge
linksharing
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.192125578s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:39:02 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.192158771s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
21.79538ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:39:01 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 21.836879ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [....X............................................XX.........] duration:
3.478425541s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:38:52 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.478577039s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
263.549099ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:38:49 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 263.576701ms
edge
gateway
vhost
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.122007493s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:38:48 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.12203201s
edge
authservice
http
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.355198161s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:38:45 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.355235441s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
253.841213ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:38:44 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 253.868425ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
27.594302ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:38:35 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 27.618228ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.902776444s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:38:31 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.902809647s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.....X............................................XX........] duration:
3.494443741s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:38:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.494497883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
4.916749685s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:38:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.916798878s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
260.910063ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:38:23 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 260.941523ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......X.XX.....X......................................X....] duration:
5.910125246s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:38:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.910153339s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
22.335129ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:38:19 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.358554ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.........X................................................X.] duration:
484.991832ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:38:15 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 485.010096ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.762948976s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:38:13 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.762983943s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.095593519s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:38:12 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.095625078s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
270.694483ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:38:11 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 270.722297ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
241.897941ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:38:11 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 241.925845ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
244.968538ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:38:05 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 244.998014ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.182590545s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:38:02 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.182622367s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [......X............................................XX.......] duration:
3.471586584s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:38:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.471633202s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.153283345s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:38:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.153314664s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
22.45209ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:37:57 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.479695ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.82419353s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:37:55 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.824334058s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
249.017748ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:37:51 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 249.037607ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
264.179098ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:37:50 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 264.202803ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
22.946423ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:37:50 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.970799ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
266.958541ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:37:43 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 266.983187ms
edge
linksharing
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.178478052s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:37:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.178506847s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
245.676888ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:37:33 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 245.702476ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
28.825307ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:37:24 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 28.85322ms
edge
linksharing
bgp
ap1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.115124345s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:37:15 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.11514845s
edge
authservice
http
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [........X.XX.....X......................................X...] duration:
5.953997644s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:37:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.954034885s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.828276357s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:37:06 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.82829928s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.......X............................................XX......] duration:
3.410289206s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:37:01 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.410313894s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.10252228s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:37:00 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.102556565s
edge
authservice
http
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.385149613s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:36:59 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.385178097s
core
uplink
eu1
|