Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...................X........................................] duration:
259.202609ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:59:57 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 259.224861ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
19.315619ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:59:57 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 19.345726ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
476.517218ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:59:55 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 476.545483ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
243.259471ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:59:42 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 243.37455ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [.....................XX................X.........X.X...X.X.X] duration:
5.001077661s trace:
https://jaeger.dev.storj.io/trace/df97ab9c1b2ebbd
FAIL (2025-12-09 00:59:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001101196s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....................X.......................................] duration:
263.029296ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:59:37 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 263.055826ms
edge
linksharing
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.1773882s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:59:36 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.177418789s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.313897648s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:59:34 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 2.31392512s
edge
authservice
http
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.60468593s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:59:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.604714864s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X..X...................] duration:
5.049218652s trace:
https://jaeger.dev.storj.io/trace/44eba3f11f79732e
FAIL (2025-12-09 00:59:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.049251323s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [......................XX................X.........X.X...X.X.] duration:
3.509461867s trace:
https://jaeger.dev.storj.io/trace/2f945c3bec73be19
FAIL (2025-12-09 00:59:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.509488278s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [......................XX....................................] duration:
275.325239ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:59:21 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 275.352402ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
472.136097ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:59:17 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 472.227129ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
105.699859ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:59:11 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 105.730788ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................X..X..................] duration:
4.971485355s trace:
https://jaeger.dev.storj.io/trace/44eba3f11f79732e
FAIL (2025-12-09 00:59:07 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.97150407s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.......................XX................X.........X.X...X.X] duration:
5.001109422s trace:
https://jaeger.dev.storj.io/trace/2f945c3bec73be19
FAIL (2025-12-09 00:59:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001212389s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
248.224175ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:59:05 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 248.254423ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
22.299502ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:59:04 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.324429ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
242.32408ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:58:56 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 242.35528ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.842984918s trace:
https://jaeger.dev.storj.io/trace/16f5f0214b7e09da
FAIL (2025-12-09 00:58:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.056163163 (81h17m12.927246198s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.057949975 (81h17m12.92903305s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059319273 (81h17m12.930402308s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059492453 (81h17m12.930575498s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059829985 (81h17m12.93091303s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060311452 (81h17m12.931394507s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060638766 (81h17m12.931721801s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.061436635 (81h17m12.93251967s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.062205830 (81h17m12.933288875s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063000051 (81h17m12.934083086s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063836003 (81h17m12.934919058s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064692454 (81h17m12.935775499s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064959161 (81h17m12.936042216s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066239960 (81h17m12.937323015s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066383043 (81h17m12.937466108s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067300530 (81h17m12.938383575s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067455816 (81h17m12.938538861s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:14.495330055 (81h17m14.36641309s) [upload=71023]: Next returned nil segment
Dec 9 00:46:14.495350613 (81h17m14.366433688s) [upload=71023]: Waiting for error group managing segments...
Dec 9 00:46:14.883721607 (81h17m14.754804652s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.929183823 (81h17m14.800266868s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.941170366 (81h17m14.812253411s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.959305788 (81h17m14.830388833s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.028943597 (81h17m14.900026632s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.033566479 (81h17m14.904649514s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.059529566 (81h17m14.930612611s) [noise=true , address=77.90.31.17:4789 , piece_id=22ZFVENBTZP7MWLLHV5HSP7QESSI4P3W67O2YZ4EOUEP4GAGVD5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.065748207 (81h17m14.936831242s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.072515793 (81h17m14.943598828s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.073404145 (81h17m14.94448718s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104276279 (81h17m14.975359324s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104945043 (81h17m14.976028098s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.112208182 (81h17m14.983291217s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115153711 (81h17m14.986236756s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115171264 (81h17m14.986254309s) [seg_pos={0 0}, upload=71023]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:46:15.115426842 (81h17m14.986509887s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNMd8YrgaagKDoWivhbTmRc5TPM6BNiHscECBF2yhncKgkvz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115435798 (81h17m14.986518843s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115498558 (81h17m14.986581613s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N9N1pcaVn3nPh1WY7uktz8Z7fwJGqnTdVwZMAW5SVrqvRA8VV): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115517804 (81h17m14.986600859s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115593578 (81h17m14.986676623s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a3tsVj8R78geQJnTzdSroG6H6RiW3xv5YLyiBh1ctwSvAKeRz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115631711 (81h17m14.986714766s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115853494 (81h17m14.986936539s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EPCQmYb816aegE9vPSoJYByMUEvhP4DUiabis2gh9QjXQC3s): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115862621 (81h17m14.986945676s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115922986 (81h17m14.987006031s) [seg_pos={0 0}, upload=71023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:46:15.116025181 (81h17m14.987108226s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:46:15.116032685 (81h17m14.98711573s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:47:25.980830692 (81h18m25.851913727s) [upload=71023]: Done uploading segments...
Dec 9 00:47:25.980980237 (81h18m25.852063282s) [upload=71023]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.843005687s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [.......................XX...................................] duration:
263.71389ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:58:52 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 263.740149ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.....................X......................................] duration:
260.910051ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:58:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 260.931552ms
edge
linksharing
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
1.085976587s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:58:43 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.086005383s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
470.96689ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:58:41 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 470.999771ms
core
satellite
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.164227199s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:58:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.164250693s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=6) [........................XX................X.........X.X...X.] duration:
3.41017277s trace:
https://jaeger.dev.storj.io/trace/3724062551e75c4b
FAIL (2025-12-09 00:58:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.4101992s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.132752596s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:58:25 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.132773375s
edge
authservice
http
bgp
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [X......................................X..X.................] duration:
5.144624797s trace:
https://jaeger.dev.storj.io/trace/44eba3f11f79732e
FAIL (2025-12-09 00:58:22 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.144651668s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
26.622754ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:58:22 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 26.65234ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.930292916s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:58:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.930319558s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.888289421s trace:
https://jaeger.dev.storj.io/trace/16f5f0214b7e09da
FAIL (2025-12-09 00:58:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.056163163 (81h17m12.927246198s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.057949975 (81h17m12.92903305s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059319273 (81h17m12.930402308s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059492453 (81h17m12.930575498s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059829985 (81h17m12.93091303s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060311452 (81h17m12.931394507s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060638766 (81h17m12.931721801s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.061436635 (81h17m12.93251967s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.062205830 (81h17m12.933288875s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063000051 (81h17m12.934083086s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063836003 (81h17m12.934919058s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064692454 (81h17m12.935775499s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064959161 (81h17m12.936042216s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066239960 (81h17m12.937323015s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066383043 (81h17m12.937466108s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067300530 (81h17m12.938383575s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067455816 (81h17m12.938538861s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:14.495330055 (81h17m14.36641309s) [upload=71023]: Next returned nil segment
Dec 9 00:46:14.495350613 (81h17m14.366433688s) [upload=71023]: Waiting for error group managing segments...
Dec 9 00:46:14.883721607 (81h17m14.754804652s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.929183823 (81h17m14.800266868s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.941170366 (81h17m14.812253411s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.959305788 (81h17m14.830388833s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.028943597 (81h17m14.900026632s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.033566479 (81h17m14.904649514s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.059529566 (81h17m14.930612611s) [noise=true , address=77.90.31.17:4789 , piece_id=22ZFVENBTZP7MWLLHV5HSP7QESSI4P3W67O2YZ4EOUEP4GAGVD5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.065748207 (81h17m14.936831242s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.072515793 (81h17m14.943598828s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.073404145 (81h17m14.94448718s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104276279 (81h17m14.975359324s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104945043 (81h17m14.976028098s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.112208182 (81h17m14.983291217s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115153711 (81h17m14.986236756s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115171264 (81h17m14.986254309s) [seg_pos={0 0}, upload=71023]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:46:15.115426842 (81h17m14.986509887s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNMd8YrgaagKDoWivhbTmRc5TPM6BNiHscECBF2yhncKgkvz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115435798 (81h17m14.986518843s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115498558 (81h17m14.986581613s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N9N1pcaVn3nPh1WY7uktz8Z7fwJGqnTdVwZMAW5SVrqvRA8VV): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115517804 (81h17m14.986600859s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115593578 (81h17m14.986676623s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a3tsVj8R78geQJnTzdSroG6H6RiW3xv5YLyiBh1ctwSvAKeRz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115631711 (81h17m14.986714766s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115853494 (81h17m14.986936539s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EPCQmYb816aegE9vPSoJYByMUEvhP4DUiabis2gh9QjXQC3s): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115862621 (81h17m14.986945676s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115922986 (81h17m14.987006031s) [seg_pos={0 0}, upload=71023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:46:15.116025181 (81h17m14.987108226s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:46:15.116032685 (81h17m14.98711573s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:47:25.980830692 (81h18m25.851913727s) [upload=71023]: Done uploading segments...
Dec 9 00:47:25.980980237 (81h18m25.852063282s) [upload=71023]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.888318387s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
470.449033ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:58:06 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 470.477999ms
core
satellite
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.162947543s trace:
https://jaeger.dev.storj.io/trace/39dda67c5a7ef46d
FAIL (2025-12-09 00:58:04 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.163057934s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=6) [.........................XX................X.........X.X...X] duration:
5.001135123s trace:
https://jaeger.dev.storj.io/trace/3724062551e75c4b
FAIL (2025-12-09 00:57:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001162817s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
250.27508ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/64025b92da229a61
FAIL (2025-12-09 00:57:57 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": dial tcp 207.211.208.129:443: i/o timeout
Duration: 250.333501ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
246.318788ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:57:54 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 246.345267ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
22.074463ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:57:54 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 22.097969ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X......................................X..X................] duration:
5.206036225s trace:
https://jaeger.dev.storj.io/trace/44eba3f11f79732e
FAIL (2025-12-09 00:57:45 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.2060603s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
261.212016ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:57:41 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 261.243246ms
edge
linksharing
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..........................XX................X.........X.X...] duration:
3.510241823s trace:
https://jaeger.dev.storj.io/trace/49e6bef034337537
FAIL (2025-12-09 00:57:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.510269946s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
2 out of 60 is failed (threshold=3) [........................XX..................................] duration:
262.989121ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/45c08147c7083bce
FAIL (2025-12-09 00:57:40 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout
Duration: 263.018156ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.981840818s trace:
https://jaeger.dev.storj.io/trace/29b49b34e5929e2f
FAIL (2025-12-09 00:57:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.981865745s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...........................X................................] duration:
730.8788ms target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/200ebca2a36ec82e
FAIL (2025-12-09 00:57:34 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 730.903458ms
edge
linksharing
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.090691126s trace:
https://jaeger.dev.storj.io/trace/2744e7e74333fadd
FAIL (2025-12-09 00:57:26 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout
Duration: 1.090716805s
edge
authservice
http
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
55.218816ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/d6823dba42570ec
FAIL (2025-12-09 00:57:25 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout
Duration: 55.244975ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
466.774718ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/6b1ae90bcdd087c7
FAIL (2025-12-09 00:57:24 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 466.806308ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.881717342s trace:
https://jaeger.dev.storj.io/trace/16f5f0214b7e09da
FAIL (2025-12-09 00:57:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.056163163 (81h17m12.927246198s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.057949975 (81h17m12.92903305s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059319273 (81h17m12.930402308s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059492453 (81h17m12.930575498s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.059829985 (81h17m12.93091303s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060311452 (81h17m12.931394507s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.060638766 (81h17m12.931721801s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.061436635 (81h17m12.93251967s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.062205830 (81h17m12.933288875s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063000051 (81h17m12.934083086s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.063836003 (81h17m12.934919058s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064692454 (81h17m12.935775499s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.064959161 (81h17m12.936042216s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066239960 (81h17m12.937323015s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.066383043 (81h17m12.937466108s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067300530 (81h17m12.938383575s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:13.067455816 (81h17m12.938538861s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Uploading piece...
Dec 9 00:46:14.495330055 (81h17m14.36641309s) [upload=71023]: Next returned nil segment
Dec 9 00:46:14.495350613 (81h17m14.366433688s) [upload=71023]: Waiting for error group managing segments...
Dec 9 00:46:14.883721607 (81h17m14.754804652s) [noise=true , address=64.49.13.155:4914 , piece_id=IWMWBIGHFIMSSDWLUNLPW635SUZYL5QUXBBDVT4Q7POXKUW4WJ5Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.929183823 (81h17m14.800266868s) [noise=true , address=145.79.227.24:4584 , piece_id=I2P5HMRRE234SLOKTQTBRXC5BJDACGFSZ4C3QDIM3R7H7VOLAR6Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.941170366 (81h17m14.812253411s) [noise=true , address=145.79.227.33:4874 , piece_id=4FSL6J7KRQIXTAC2R7T5SDHYMSFGLFL3SUBLFOYYRZM47V3G73OQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:14.959305788 (81h17m14.830388833s) [noise=true , address=64.49.13.154:4934 , piece_id=6TISGSS4P6CP6FJL7DFWUXYT5X6KQJ4OYHSIZGSMQV4RZMFTGU4A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.028943597 (81h17m14.900026632s) [noise=true , address=5.231.73.4:4003 , piece_id=4K3ONY56R3TZYJZI5OJ4ADJXUALZPTHP236QDFDRNZUNI3LRIADA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.033566479 (81h17m14.904649514s) [noise=true , address=77.90.31.16:4738 , piece_id=TCG6E2SKWNQ3R23XJEJG45H4WKPYHFH57SM3JLFMNXDI455T5D5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.059529566 (81h17m14.930612611s) [noise=true , address=77.90.31.17:4789 , piece_id=22ZFVENBTZP7MWLLHV5HSP7QESSI4P3W67O2YZ4EOUEP4GAGVD5A, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.065748207 (81h17m14.936831242s) [noise=true , address=45.91.116.4:4956 , piece_id=QQ6LRHTMTGEBHNUH3HX545OSX2D4W57ZJH2HE4ULX3TSRZKCSJWQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.072515793 (81h17m14.943598828s) [noise=true , address=77.90.31.10:4534 , piece_id=I267OI6FD4ZA354UR7JZNXZBIYR6HJ44IPX2AHV2CQRO5RYBHFHQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.073404145 (81h17m14.94448718s) [noise=true , address=45.91.116.5:4327 , piece_id=M55GD6NYM7XLFPX2PWUZBJTKBCIID63SDCRTAINR4K3QXHQ7VFWA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104276279 (81h17m14.975359324s) [noise=true , address=45.91.116.11:4147 , piece_id=Z5BCQU3TM45AVD4KSHQ74AFDEBMALRUD3PXQPQEV54CBLYL3TR3Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.104945043 (81h17m14.976028098s) [noise=true , address=45.91.116.13:4187 , piece_id=TZEUTEYC4MWHZPOILSHTWPLNAQWKVNEM4JYGOYM54TOIH6MES2WA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.112208182 (81h17m14.983291217s) [noise=true , address=45.91.116.6:4983 , piece_id=UWFWT4HF5IFO25OED2UEZSFHFV7TZBSTUETBC2LQ7EUQLG4SUD2Q, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115153711 (81h17m14.986236756s) [noise=true , address=45.91.116.12:4254 , piece_id=V6Z3FY3IX4ZMXDSQWYJD7XRSHH7TF5PO3O2WGAP37H5QWR23ZVAQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: <nil>
Dec 9 00:46:15.115171264 (81h17m14.986254309s) [seg_pos={0 0}, upload=71023]: Segment reached optimal threshold of 14 pieces.
Dec 9 00:46:15.115426842 (81h17m14.986509887s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNMd8YrgaagKDoWivhbTmRc5TPM6BNiHscECBF2yhncKgkvz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115435798 (81h17m14.986518843s) [noise=true , address=145.79.227.30:4783 , piece_id=36FPQMRRBG3OZW3R3DGEOLYTJZK5M4RBGVUU5K25N2IY6UVFHLGQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115498558 (81h17m14.986581613s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12N9N1pcaVn3nPh1WY7uktz8Z7fwJGqnTdVwZMAW5SVrqvRA8VV): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115517804 (81h17m14.986600859s) [noise=true , address=77.90.31.13:4639 , piece_id=7NXF7N5A43FAF2K5FPKP224XVCUNDA2CHAKEYDSBVAMJDYHYXACQ, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115593578 (81h17m14.986676623s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a3tsVj8R78geQJnTzdSroG6H6RiW3xv5YLyiBh1ctwSvAKeRz): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115631711 (81h17m14.986714766s) [noise=true , address=77.90.31.10:4535 , piece_id=3PGFYM3GJWZ5XSV7ESR4EPIOYKZ7QA4Q4DUKHMFKUTJMUDIFW5AA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115853494 (81h17m14.986936539s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17EPCQmYb816aegE9vPSoJYByMUEvhP4DUiabis2gh9QjXQC3s): protocol: expected piece hash; context canceled; EOF
Dec 9 00:46:15.115862621 (81h17m14.986945676s) [noise=true , address=45.91.116.8:4065 , piece_id=CADF7N2FM2G2JBFQPZOOGNXO47RWIIPHWBW6BEZI65IVAHUV2IQA, seg_pos={0 0}, upload=71023]: Long tail cancellation due to optimal threshold
Dec 9 00:46:15.115922986 (81h17m14.987006031s) [seg_pos={0 0}, upload=71023]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Dec 9 00:46:15.116025181 (81h17m14.987108226s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 00:46:15.116032685 (81h17m14.98711573s) [upload=71023]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 00:47:25.980830692 (81h18m25.851913727s) [upload=71023]: Done uploading segments...
Dec 9 00:47:25.980980237 (81h18m25.852063282s) [upload=71023]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.881757588s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
272.334574ms target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/dd0fa7a1ae6bdc0
FAIL (2025-12-09 00:57:10 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 272.357791ms
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........................X.................................] duration:
36.035398ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/af91489e18f8dd7
FAIL (2025-12-09 00:57:06 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout
Duration: 36.054213ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [..X......................................X..X...............] duration:
4.742859476s trace:
https://jaeger.dev.storj.io/trace/44eba3f11f79732e
FAIL (2025-12-09 00:57:03 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 4.74289212s
core
uplink
eu1
|