Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
18.316478ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:35:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 18.345333ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
261.495468ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:35:44 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 261.520507ms
edge
gateway
vhost
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.201863206s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:35:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.201888365s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
484.381599ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735
FAIL (2025-12-09 00:35:33 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 484.413089ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
261.304194ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:35:32 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 261.33293ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
249.031404ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:35:31 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 249.064998ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
20.94456ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:35:31 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 20.971634ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
242.382466ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:35:22 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 242.414957ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.874499369s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:35:19 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.874531249s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
4.869855628s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:35:17 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.869884372s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
24.399819ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:35:13 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 24.428044ms
edge
linksharing
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [..........X.XX.....X......................................X.] duration:
5.804895537s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:35:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.804913912s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.131313251s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:35:09 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.131353317s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
273.330383ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:35:05 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 273.358516ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [...........X.XX.....X......................................X] duration:
1m0.000643501s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:35:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000667517s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [X.........X............................................XX...] duration:
3.406463672s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:34:59 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.406480133s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.179645362s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:34:55 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.179672063s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
275.756522ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:34:53 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 275.779785ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
21.815469ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:34:43 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 21.846498ms
edge
linksharing
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [.X.........X............................................XX..] duration:
3.395779939s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:34:41 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.395805308s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
245.258681ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:34:39 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 245.283059ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.742158388s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:34:31 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.742184128s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
4.913867277s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:34:29 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.913897082s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
24.207374ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:34:29 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 24.230366ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
484.88654ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735
FAIL (2025-12-09 00:34:28 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 484.934513ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
242.203694ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:34:28 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 242.233632ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
273.096717ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:34:26 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 273.125301ms
edge
gateway
vhost
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.128948309s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:34:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.12897461s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
241.993645ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:34:04 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 242.023732ms
edge
linksharing
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.106779883s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:34:01 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.106812116s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
257.757029ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:33:59 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 257.835048ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
253.565208ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:33:59 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 253.596247ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
23.900058ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:33:52 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 23.924074ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.883870855s trace:
https://jaeger.dev.storj.io/trace/221485e4375dc5e9
FAIL (2025-12-09 00:33:46 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.273496693 (80h43m57.144579738s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.276438745 (80h43m57.147521791s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.277350421 (80h43m57.148433466s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.278547981 (80h43m57.149631016s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.279127134 (80h43m57.150210169s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.284610114 (80h43m57.15569316s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.288373360 (80h43m57.159456405s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.290400481 (80h43m57.161483516s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292482585 (80h43m57.16356565s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.292569771 (80h43m57.163652826s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.294155992 (80h43m57.165239027s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.295419147 (80h43m57.166502192s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.298123437 (80h43m57.169206482s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.299130625 (80h43m57.17021367s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302603368 (80h43m57.173686413s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:57.302742894 (80h43m57.173825939s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Uploading piece...
Dec 9 00:12:58.757768352 (80h43m58.628851387s) [upload=70524]: Next returned nil segment
Dec 9 00:12:58.757780825 (80h43m58.62886386s) [upload=70524]: Waiting for error group managing segments...
Dec 9 00:12:59.167428791 (80h43m59.038511826s) [noise=true , address=145.79.227.41:4108 , piece_id=ZIVSIG5HIT4WWFNATU2RTLJBBDYKETDIXROEI7UL2MLXLEZ6D2RA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202245459 (80h43m59.073328494s) [noise=true , address=145.79.227.29:4961 , piece_id=TQPNWGGNMFCUINRM35DB5RWU35VRZYG4GPMYZSIVG4SQQMSE4U7A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202524801 (80h43m59.073607846s) [noise=true , address=145.79.227.17:4986 , piece_id=YQYGQULZFH452JA5IODKYPC3LETX7T357O5ALAKDWQMU5EJCLLBQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.202790055 (80h43m59.07387309s) [noise=true , address=145.79.227.51:4414 , piece_id=KGHPUOW2DFXDVLDTK6EVH24UBLADTDDBRRE3LVXYZNLHUDCH2JOQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.224642217 (80h43m59.095725252s) [noise=true , address=145.79.227.4:4188 , piece_id=E4242ODRWKUCIQIQFDYVZBZJF6HRHIY6KFJR7T7L2LTUL2NLHC6A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.248203811 (80h43m59.119286856s) [noise=true , address=77.90.31.13:4630 , piece_id=ZEGIG2MES6L7FQJF443D6KM2VQJEM3C75MXNMW2A3XMQX2RRVKTA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.250778595 (80h43m59.12186164s) [noise=true , address=77.90.31.14:4672 , piece_id=7YAXWDIIXYRD42GSBSWKKMKRCJB6YPIEQFJC37YP4NO2DL7U4Z5A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.266778802 (80h43m59.137861838s) [noise=true , address=77.90.31.12:4618 , piece_id=3QY2MK56ALE4VZHEC6VAYJHAXPEGC5G7MNW52V4TVVJXQNS5HQEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.286083301 (80h43m59.157166336s) [noise=true , address=45.91.116.11:4131 , piece_id=6KKROCQQMGZIIKKWAZNQ6PP6CHVWTULFOFYVRBB4VV6XMQVOBQ4A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.323963693 (80h43m59.195046728s) [noise=true , address=45.91.116.10:4112 , piece_id=RHXAFNDD3LSI4YZ7KMOCQVEROYTTJ6MDFDINMCNCK5LOBEOTUXKQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.328753564 (80h43m59.199836609s) [noise=true , address=77.90.31.21:4887 , piece_id=EEXX5CQ65UUHP2IFFKU5R5C6XTFUEY7G4NOYXQQL6XILA7WZPNWQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.329322737 (80h43m59.200405772s) [noise=true , address=45.91.116.6:4000 , piece_id=ENQFG6S3ZEWLXQSAC554I5N75XDDJSI3K4PSIILZTFISLWK7PJGA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.330595620 (80h43m59.201678656s) [noise=true , address=77.90.31.11:4563 , piece_id=ZGHYIVTFORGVUQIVWOV5IH7VPMUJGGVCMGEUXGFMRJPCRL2BFHNA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342437661 (80h43m59.213520706s) [noise=true , address=45.91.116.7:4027 , piece_id=GRCU7C6TUT7VAQ7B7PL3LUD4IEVQWKICWYRLFEMKGNF4LVP2C33A, seg_pos={0 0}, upload=70524]: Done uploading piece. err: <nil>
Dec 9 00:12:59.342464372 (80h43m59.213547417s) [seg_pos={0 0}, upload=70524]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:12:59.342753241 (80h43m59.213836276s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rfuAG41swAn82EM5ZTAN31NhGp4UKMHFiZiT57NXmSAi3MHe2): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342768691 (80h43m59.213851726s) [noise=true , address=45.91.116.12:4229 , piece_id=QSZCZSFNEK3J6KN73FDNS6X72LM323NTM4K7IRP3Z5MLWWTR3MEA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342800302 (80h43m59.213883347s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nY93g6WbZs2rfBYoKtf4Z7xGSr1QwizrBibKjpgKFbTbnVe1c): protocol: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58774->145.79.227.40:4056: use of closed network connection
Dec 9 00:12:59.342811893 (80h43m59.213894928s) [noise=true , address=145.79.227.40:4056 , piece_id=EZC2HIYLMJ4V5BKBOAFB2PX3W5WMDH4TBTKJPPCY2CTIPDBSJEOA, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.342901854 (80h43m59.213984889s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12d12pAMGhxjtLBE1tp9yEE7NMdfh8kcvsA8abhTnM8nbBB5LEK): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.342911423 (80h43m59.213994468s) [noise=true , address=45.91.116.14:4206 , piece_id=45TLUVKXRSH44QOP34JPYO6HZTRDGTGTJEVNFPSFKZT2O6MYV4OQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343003398 (80h43m59.214086433s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12niHLkT5pbmYSbarHZGeeCRtwjZRXtvEtHMGAb7ja4a7Xuuvfi): protocol: expected piece hash; context canceled; EOF
Dec 9 00:12:59.343056940 (80h43m59.214139995s) [noise=true , address=145.79.227.44:4197 , piece_id=WSTR4ASYVX2B4VZ7BME4QHSJ6SWOKRJSIP7R3WJFZYLBXKDQUFQQ, seg_pos={0 0}, upload=70524]: Long tail cancellation due to optimal threshold
Dec 9 00:12:59.343082218 (80h43m59.214165253s) [seg_pos={0 0}, upload=70524]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:12:59.343167510 (80h43m59.214250555s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:12:59.343204701 (80h43m59.214287766s) [upload=70524]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:13:42.867239922 (80h44m42.738322967s) [upload=70524]: Done uploading segments...
Dec 9 00:13:42.867266332 (80h44m42.738349377s) [upload=70524]: Upload finished. err: <nil>
Duration: 5.8839051s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.261419365s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:33:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.261466104s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
271.536928ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:33:42 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 271.553288ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
268.848348ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:33:41 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 268.876772ms
edge
gateway
vhost
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.200997268s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:33:40 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.201044267s
edge
authservice
http
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
243.062318ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:33:32 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 243.08904ms
edge
gateway
vhost
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.15596936s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:33:32 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.155989759s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.081317298s remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7fc0ad6d99ae5735
FAIL (2025-12-09 00:33:31 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 5.081344661s
core
satellite
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
22.628867ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:33:31 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.662081ms
edge
gateway
vhost
bgp
ap1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000870957s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:33:30 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 30.000897068s
edge
authservice
http
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [..X.........X............................................XX.] duration:
3.477968196s trace:
https://jaeger.dev.storj.io/trace/576248d76c2fe5c4
FAIL (2025-12-09 00:33:27 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.477997561s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..] duration:
262.014786ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:33:25 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 262.042889ms
edge
linksharing
bgp
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
4.923458554s trace:
https://jaeger.dev.storj.io/trace/7bce72a308ed627a
FAIL (2025-12-09 00:33:16 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.923485094s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
243.73586ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:33:15 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 243.759307ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
30.001261191s target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:33:14 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 30.001282872s
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............X.XX.....X......................................] duration:
5.837466474s trace:
https://jaeger.dev.storj.io/trace/3ddc3b2546ea0c66
FAIL (2025-12-09 00:33:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.837489266s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
31.308996ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:33:01 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 31.34237ms
edge
gateway
vhost
bgp
ap1
|