Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...........X...............X................................] duration:
5.747687384s trace:
https://jaeger.dev.storj.io/trace/194dcfc888c86a7
FAIL (2025-12-08 04:31:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.747719134s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
7.260526708s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:31:22 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 7.260558969s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.............................................XX.............] duration:
33.466566ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:31:22 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 33.519056ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.X.................................X........................] duration:
2.391631418s remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:31:07 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 2.391664831s
core
satellite
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.828806878s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:31:00 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.828835351s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.782666699s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:30:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.782773402s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..............................................XX............] duration:
23.667667ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:30:43 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 23.718785ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..X.................................X.......................] duration:
953.869609ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:30:41 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 953.898725ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [............X...............X...............................] duration:
5.825595314s trace:
https://jaeger.dev.storj.io/trace/194dcfc888c86a7
FAIL (2025-12-08 04:30:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.825637312s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.727285303s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:30:34 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.727310311s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.747210464s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:30:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.747408951s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [..X...X.X..XX.............X....X........X............X.X....] duration:
3.464647913s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:30:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.464802768s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [...X...X.X..XX.............X....X........X............X.X...] duration:
3.506428752s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:30:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.506461623s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.012858386s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:30:01 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.012896307s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...............................................XX...........] duration:
22.303851ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:29:57 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 22.32487ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...X.................................X......................] duration:
879.063145ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:29:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 879.091198ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.972274867s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:29:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.972306636s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [................................................XX..........] duration:
27.165898ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:29:36 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 27.191388ms
edge
linksharing
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.............X...............X..............................] duration:
5.818997529s trace:
https://jaeger.dev.storj.io/trace/194dcfc888c86a7
FAIL (2025-12-08 04:29:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.819025151s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.807906088s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:29:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.807933398s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [....X...X.X..XX.............X....X........X............X.X..] duration:
3.513233709s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:29:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.513263927s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.................................................XX.........] duration:
22.688483ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:29:06 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 22.735082ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.990657741s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:28:58 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.990811294s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
6.089279574s trace:
https://jaeger.dev.storj.io/trace/65a469d74c42aea
FAIL (2025-12-08 04:28:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:25:18.914773219 (60h56m18.785856264s) [upload=53478]: Starting upload
Dec 8 04:25:18.914857621 (60h56m18.785940666s) [upload=53478]: Done starting upload
Dec 8 04:25:18.915137754 (60h56m18.786220799s) [upload=53478]: Uploading segments...
Dec 8 04:25:18.915241051 (60h56m18.786324096s) [upload=53478]: Got next segment. Inline: false
Dec 8 04:25:18.915277690 (60h56m18.786360735s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:25:18.915283842 (60h56m18.786366897s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:25:20.019106339 (60h56m19.890189384s) [seg_pos={0 0}, upload=53478]: Begin upload segment...
Dec 8 04:25:20.019211690 (60h56m19.890294725s) [seg_pos={0 0}, upload=53478]: Done begin upload segment.
Dec 8 04:25:20.019268368 (60h56m19.890351403s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019471163 (60h56m19.890554219s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019945106 (60h56m19.891028151s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020309811 (60h56m19.891392856s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020601055 (60h56m19.891684101s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021049069 (60h56m19.892132114s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021522419 (60h56m19.892605465s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022053612 (60h56m19.893136647s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022597697 (60h56m19.893680742s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024691555 (60h56m19.89577459s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024902927 (60h56m19.895985972s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.026445404 (60h56m19.897528449s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.028229382 (60h56m19.899312417s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.029879874 (60h56m19.900962909s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.031127490 (60h56m19.902210535s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.032447524 (60h56m19.903530569s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:21.774559768 (60h56m21.645642813s) [upload=53478]: Next returned nil segment
Dec 8 04:25:21.774575198 (60h56m21.645658253s) [upload=53478]: Waiting for error group managing segments...
Dec 8 04:25:22.055105228 (60h56m21.926188263s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061412458 (60h56m21.932495493s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061628369 (60h56m21.932711414s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.067549273 (60h56m21.938632308s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.087910595 (60h56m21.95899364s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.094683903 (60h56m21.965766948s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.106150907 (60h56m21.977233952s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.107865303 (60h56m21.978948348s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.196447879 (60h56m22.067530914s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.270254411 (60h56m22.141337456s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.424993952 (60h56m22.296076997s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.428855525 (60h56m22.29993856s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429349886 (60h56m22.300432932s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429359846 (60h56m22.300442891s) [seg_pos={0 0}, upload=53478]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:25:22.429515522 (60h56m22.300598567s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429522505 (60h56m22.30060555s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.429760599 (60h56m22.300843644s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429767552 (60h56m22.300850587s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430291820 (60h56m22.301374865s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.430345142 (60h56m22.301428187s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430391189 (60h56m22.301474234s) [seg_pos={0 0}, upload=53478]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:25:22.430521177 (60h56m22.301604222s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:25:22.430528441 (60h56m22.301611486s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:27:43.788063719 (60h58m43.659146764s) [upload=53478]: Done uploading segments...
Dec 8 04:27:43.788124595 (60h58m43.65920764s) [upload=53478]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.089308209s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [....X.................................X.....................] duration:
841.843562ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:28:52 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 841.871395ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..................................................XX........] duration:
24.724499ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:28:39 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 24.751331ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.564762749s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:28:33 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.564794179s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.768275525s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:28:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.768303809s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [.....X...X.X..XX.............X....X........X............X.X.] duration:
3.56410673s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:28:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.564137279s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.....X.................................X....................] duration:
695.035773ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:28:29 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 695.064998ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..............X...............X.............................] duration:
6.051180988s trace:
https://jaeger.dev.storj.io/trace/194dcfc888c86a7
FAIL (2025-12-08 04:28:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.051208329s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.835878985s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:28:06 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.835978174s
core
uplink
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [......X.................................X...................] duration:
943.055458ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:27:57 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 943.083903ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
8.037933895s trace:
https://jaeger.dev.storj.io/trace/65a469d74c42aea
FAIL (2025-12-08 04:27:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:25:18.914773219 (60h56m18.785856264s) [upload=53478]: Starting upload
Dec 8 04:25:18.914857621 (60h56m18.785940666s) [upload=53478]: Done starting upload
Dec 8 04:25:18.915137754 (60h56m18.786220799s) [upload=53478]: Uploading segments...
Dec 8 04:25:18.915241051 (60h56m18.786324096s) [upload=53478]: Got next segment. Inline: false
Dec 8 04:25:18.915277690 (60h56m18.786360735s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:25:18.915283842 (60h56m18.786366897s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:25:20.019106339 (60h56m19.890189384s) [seg_pos={0 0}, upload=53478]: Begin upload segment...
Dec 8 04:25:20.019211690 (60h56m19.890294725s) [seg_pos={0 0}, upload=53478]: Done begin upload segment.
Dec 8 04:25:20.019268368 (60h56m19.890351403s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019471163 (60h56m19.890554219s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019945106 (60h56m19.891028151s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020309811 (60h56m19.891392856s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020601055 (60h56m19.891684101s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021049069 (60h56m19.892132114s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021522419 (60h56m19.892605465s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022053612 (60h56m19.893136647s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022597697 (60h56m19.893680742s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024691555 (60h56m19.89577459s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024902927 (60h56m19.895985972s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.026445404 (60h56m19.897528449s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.028229382 (60h56m19.899312417s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.029879874 (60h56m19.900962909s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.031127490 (60h56m19.902210535s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.032447524 (60h56m19.903530569s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:21.774559768 (60h56m21.645642813s) [upload=53478]: Next returned nil segment
Dec 8 04:25:21.774575198 (60h56m21.645658253s) [upload=53478]: Waiting for error group managing segments...
Dec 8 04:25:22.055105228 (60h56m21.926188263s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061412458 (60h56m21.932495493s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061628369 (60h56m21.932711414s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.067549273 (60h56m21.938632308s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.087910595 (60h56m21.95899364s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.094683903 (60h56m21.965766948s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.106150907 (60h56m21.977233952s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.107865303 (60h56m21.978948348s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.196447879 (60h56m22.067530914s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.270254411 (60h56m22.141337456s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.424993952 (60h56m22.296076997s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.428855525 (60h56m22.29993856s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429349886 (60h56m22.300432932s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429359846 (60h56m22.300442891s) [seg_pos={0 0}, upload=53478]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:25:22.429515522 (60h56m22.300598567s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429522505 (60h56m22.30060555s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.429760599 (60h56m22.300843644s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429767552 (60h56m22.300850587s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430291820 (60h56m22.301374865s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.430345142 (60h56m22.301428187s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430391189 (60h56m22.301474234s) [seg_pos={0 0}, upload=53478]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:25:22.430521177 (60h56m22.301604222s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:25:22.430528441 (60h56m22.301611486s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:27:43.788063719 (60h58m43.659146764s) [upload=53478]: Done uploading segments...
Dec 8 04:27:43.788124595 (60h58m43.65920764s) [upload=53478]: Upload finished. err: metaclient: context deadline exceeded
Duration: 8.037951749s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
2m30.740239917s trace:
https://jaeger.dev.storj.io/trace/65a469d74c42aea
FAIL (2025-12-08 04:27:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:25:18.914773219 (60h56m18.785856264s) [upload=53478]: Starting upload
Dec 8 04:25:18.914857621 (60h56m18.785940666s) [upload=53478]: Done starting upload
Dec 8 04:25:18.915137754 (60h56m18.786220799s) [upload=53478]: Uploading segments...
Dec 8 04:25:18.915241051 (60h56m18.786324096s) [upload=53478]: Got next segment. Inline: false
Dec 8 04:25:18.915277690 (60h56m18.786360735s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:25:18.915283842 (60h56m18.786366897s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:25:20.019106339 (60h56m19.890189384s) [seg_pos={0 0}, upload=53478]: Begin upload segment...
Dec 8 04:25:20.019211690 (60h56m19.890294725s) [seg_pos={0 0}, upload=53478]: Done begin upload segment.
Dec 8 04:25:20.019268368 (60h56m19.890351403s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019471163 (60h56m19.890554219s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.019945106 (60h56m19.891028151s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020309811 (60h56m19.891392856s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.020601055 (60h56m19.891684101s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021049069 (60h56m19.892132114s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.021522419 (60h56m19.892605465s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022053612 (60h56m19.893136647s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.022597697 (60h56m19.893680742s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024691555 (60h56m19.89577459s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.024902927 (60h56m19.895985972s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.026445404 (60h56m19.897528449s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.028229382 (60h56m19.899312417s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.029879874 (60h56m19.900962909s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.031127490 (60h56m19.902210535s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:20.032447524 (60h56m19.903530569s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Uploading piece...
Dec 8 04:25:21.774559768 (60h56m21.645642813s) [upload=53478]: Next returned nil segment
Dec 8 04:25:21.774575198 (60h56m21.645658253s) [upload=53478]: Waiting for error group managing segments...
Dec 8 04:25:22.055105228 (60h56m21.926188263s) [noise=true , address=145.79.227.44:4193 , piece_id=Z3RMDAHYR223TCNYY43G4WDLMPN5PNGCMSBQRVHVB75JYCA34THQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061412458 (60h56m21.932495493s) [noise=true , address=145.79.227.29:4942 , piece_id=MH7CLGJBBBSETQMIT7GZFQ7J4YL24UCXEJF3FT3362UAPH4OQ34Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.061628369 (60h56m21.932711414s) [noise=true , address=145.79.227.39:4013 , piece_id=PZORXZ7N7HA6UVTOOT3JST7J753GNPABUSUVCCE4CLDTKTGFPZSA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.067549273 (60h56m21.938632308s) [noise=true , address=145.79.227.31:4797 , piece_id=CKMWZ4SMAFM4JHFXVFQTKJ6Y2IIGNDMSDOSMZVVOSQ7OPVHTE77Q, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.087910595 (60h56m21.95899364s) [noise=true , address=145.79.227.23:4547 , piece_id=ZP2UHTODTUWI56Z3C4PPH6FWQUHJVOWASGP44QSZDL6W6UCNY4PA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.094683903 (60h56m21.965766948s) [noise=true , address=77.90.31.10:4520 , piece_id=S2EYQBGZW4BZIPGMBLMV2ITXL5FBOG6MDGXIVNETZGPKTBDUEF2A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.106150907 (60h56m21.977233952s) [noise=true , address=145.79.227.20:4438 , piece_id=PQVQFFYSPVBHPYB7TBSZZ443T652ZMNBRAZL4IPODA24P2PVWZJQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.107865303 (60h56m21.978948348s) [noise=true , address=145.79.227.42:4123 , piece_id=NE45VSHVAD34ZMNXMMF4ZQ7RKNMCEK6H32JMXDPT5HOUTPX4XVGQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.196447879 (60h56m22.067530914s) [noise=true , address=145.79.227.16:4911 , piece_id=SLOKXMX3VMEBUHYNZRNEIHUQIUI2F47XQ42RYR4NCK6INLBHSTNA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.270254411 (60h56m22.141337456s) [noise=true , address=145.79.227.33:4870 , piece_id=CUUK3MIJIKBZJRPDIQB2GKECH7DBYZJPYMNR463ZWB4R5OJJGTAQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.424993952 (60h56m22.296076997s) [noise=true , address=145.79.227.43:4158 , piece_id=5RRQKYYASS75CIASIXEQ7N6CF2KXPAH2NEHEMSNX54VYY4BN6I7A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.428855525 (60h56m22.29993856s) [noise=true , address=145.79.227.25:4617 , piece_id=NOGM2GVOST3OPDV6XDZZUPULH3TUHKNVL26UWCWQDEPGCFDSAWWQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429349886 (60h56m22.300432932s) [noise=true , address=145.79.227.31:4799 , piece_id=T4Q44CXIWFM3KM64HW27IJQR6FTJJV6HH3S5C665H7N3OFCROIHQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: <nil>
Dec 8 04:25:22.429359846 (60h56m22.300442891s) [seg_pos={0 0}, upload=53478]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:25:22.429515522 (60h56m22.300598567s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429522505 (60h56m22.30060555s) [noise=true , address=145.79.227.29:4941 , piece_id=DPPAQPV3MKJMPIFW5XB26UVPB7OZSM5YTGV5TAW5FRJD7GHA35IA, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.429760599 (60h56m22.300843644s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.429767552 (60h56m22.300850587s) [noise=true , address=145.79.227.26:4653 , piece_id=MBUD42NMO6ADMEKQAIZNH4GYWKEAL3JCDW4R65HUW2DWX7FCTP6A, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430291820 (60h56m22.301374865s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: expected piece hash; context canceled; EOF
Dec 8 04:25:22.430345142 (60h56m22.301428187s) [noise=true , address=77.90.31.10:4518 , piece_id=MEPEIZFKNE5MG6LEEGMGKL6YLLXK4NWFGQZK6DEJVVBV2XQUKSCQ, seg_pos={0 0}, upload=53478]: Long tail cancellation due to optimal threshold
Dec 8 04:25:22.430391189 (60h56m22.301474234s) [seg_pos={0 0}, upload=53478]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:25:22.430521177 (60h56m22.301604222s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:25:22.430528441 (60h56m22.301611486s) [upload=53478]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:27:43.788063719 (60h58m43.659146764s) [upload=53478]: Done uploading segments...
Dec 8 04:27:43.788124595 (60h58m43.65920764s) [upload=53478]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2m30.740267269s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.000779548s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:27:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.000808593s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [...................................................XX.......] duration:
25.859751ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:27:37 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 25.910117ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.17622301s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:27:34 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.176241015s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.95456734s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:27:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.954595733s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [......X...X.X..XX.............X....X........X............X.X] duration:
5.001445372s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:27:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001469157s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...............X...............X............................] duration:
5.836769572s trace:
https://jaeger.dev.storj.io/trace/194dcfc888c86a7
FAIL (2025-12-08 04:27:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.836824326s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.......X.................................X..................] duration:
728.766785ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:27:03 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 728.795709ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.647017809s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:26:51 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.647045641s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [........X.................................X.................] duration:
963.535714ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/47622d068430859b
FAIL (2025-12-08 04:26:44 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 963.562896ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.72960567s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:26:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.729649673s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [....................................................XX......] duration:
22.05669ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:26:36 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 22.079202ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
7.00814924s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:26:21 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 7.008179227s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.857353726s trace:
https://jaeger.dev.storj.io/trace/456cfc131726c52f
FAIL (2025-12-08 04:26:21 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.857384966s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [.......X...X.X..XX.............X....X........X............X.] duration:
3.377294407s trace:
https://jaeger.dev.storj.io/trace/33aa6929cf1b5dce
FAIL (2025-12-08 04:26:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.377320717s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.664273367s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:26:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.664305637s
edge
gateway
us-select-1
|