Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
12.500762309s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:52:12 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 12.500793467s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [......X..................................X..................] duration:
695.536516ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:52:09 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 695.583737ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
478.790873ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:52:07 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 478.82065ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.315469358s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:51:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.315497392s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.X....X........X............X.X.............................] duration:
3.507103998s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:51:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.507128965s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.......X..................................X.................] duration:
695.686842ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:51:46 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 695.71206ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.958214161s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:51:41 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.958243847s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [....................XX......................................] duration:
20.083863ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:51:41 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 20.115073ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.672803257s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:51:25 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.67283142s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..X..........................................X..............] duration:
5.666657875s trace:
https://jaeger.dev.storj.io/trace/3d9c5ba0a6d4da26
FAIL (2025-12-08 04:51:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.666684606s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
469.210501ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:51:15 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 469.254375ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [........X..................................X................] duration:
740.678044ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:51:08 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 740.705438ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.708356386s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:51:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.708388809s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..X....X........X............X.X............................] duration:
3.517806084s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:50:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.517833998s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.....................XX.....................................] duration:
34.368183ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:50:55 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 34.393782ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [...........................................X..X..........X..] duration:
5.743218963s trace:
https://jaeger.dev.storj.io/trace/327d8af62af2ad4a
FAIL (2025-12-08 04:50:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:41:10.986572683 (61h12m10.857655728s) [upload=53698]: Starting upload
Dec 8 04:41:10.986697500 (61h12m10.857780545s) [upload=53698]: Done starting upload
Dec 8 04:41:10.986782503 (61h12m10.857865538s) [upload=53698]: Uploading segments...
Dec 8 04:41:10.987063708 (61h12m10.858146753s) [upload=53698]: Got next segment. Inline: false
Dec 8 04:41:10.987116498 (61h12m10.858199543s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:41:10.987121758 (61h12m10.858204803s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:41:17.462876699 (61h12m17.333959744s) [seg_pos={0 0}, upload=53698]: Begin upload segment...
Dec 8 04:41:17.462933336 (61h12m17.334016381s) [seg_pos={0 0}, upload=53698]: Done begin upload segment.
Dec 8 04:41:17.462963864 (61h12m17.334046909s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464241417 (61h12m17.335324452s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464758772 (61h12m17.335841817s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465245028 (61h12m17.336328063s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465727617 (61h12m17.336810672s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.466300738 (61h12m17.337383783s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.467904602 (61h12m17.338987647s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.469643584 (61h12m17.340726629s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.470720414 (61h12m17.341803449s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.472841653 (61h12m17.343924708s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.474783842 (61h12m17.345866877s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.478925869 (61h12m17.350008904s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.482146932 (61h12m17.353229977s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.484118316 (61h12m17.355201361s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.486216912 (61h12m17.357299957s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.487427918 (61h12m17.358510953s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:19.196950317 (61h12m19.068033352s) [upload=53698]: Next returned nil segment
Dec 8 04:41:19.196972289 (61h12m19.068055324s) [upload=53698]: Waiting for error group managing segments...
Dec 8 04:41:19.470957375 (61h12m19.34204042s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.478965694 (61h12m19.350048739s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.488459723 (61h12m19.359542758s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.491834840 (61h12m19.362917885s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.496467461 (61h12m19.367550506s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.497643140 (61h12m19.368726185s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.513620774 (61h12m19.384703809s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.519169180 (61h12m19.390252225s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.582513759 (61h12m19.453596794s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.600844513 (61h12m19.471927549s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.627512986 (61h12m19.498596031s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.923827261 (61h12m19.794910306s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940676124 (61h12m19.811759159s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940696483 (61h12m19.811779518s) [seg_pos={0 0}, upload=53698]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:41:19.940920650 (61h12m19.812003695s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.940927292 (61h12m19.812010347s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941137082 (61h12m19.812220117s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941146079 (61h12m19.812229124s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941154806 (61h12m19.812237851s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941162220 (61h12m19.812245255s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941203298 (61h12m19.812286343s) [seg_pos={0 0}, upload=53698]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:41:19.941272510 (61h12m19.812355555s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:41:19.941277359 (61h12m19.812360404s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:50:23.376780170 (61h21m23.247863205s) [upload=53698]: Done uploading segments...
Dec 8 04:50:23.376921268 (61h21m23.248004313s) [upload=53698]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.743272224s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.........X..................................X...............] duration:
694.633288ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:50:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 694.659558ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.811446177s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:50:36 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.811473328s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [............................................X..X..........X.] duration:
5.833455296s trace:
https://jaeger.dev.storj.io/trace/327d8af62af2ad4a
FAIL (2025-12-08 04:50:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:41:10.986572683 (61h12m10.857655728s) [upload=53698]: Starting upload
Dec 8 04:41:10.986697500 (61h12m10.857780545s) [upload=53698]: Done starting upload
Dec 8 04:41:10.986782503 (61h12m10.857865538s) [upload=53698]: Uploading segments...
Dec 8 04:41:10.987063708 (61h12m10.858146753s) [upload=53698]: Got next segment. Inline: false
Dec 8 04:41:10.987116498 (61h12m10.858199543s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:41:10.987121758 (61h12m10.858204803s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:41:17.462876699 (61h12m17.333959744s) [seg_pos={0 0}, upload=53698]: Begin upload segment...
Dec 8 04:41:17.462933336 (61h12m17.334016381s) [seg_pos={0 0}, upload=53698]: Done begin upload segment.
Dec 8 04:41:17.462963864 (61h12m17.334046909s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464241417 (61h12m17.335324452s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464758772 (61h12m17.335841817s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465245028 (61h12m17.336328063s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465727617 (61h12m17.336810672s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.466300738 (61h12m17.337383783s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.467904602 (61h12m17.338987647s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.469643584 (61h12m17.340726629s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.470720414 (61h12m17.341803449s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.472841653 (61h12m17.343924708s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.474783842 (61h12m17.345866877s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.478925869 (61h12m17.350008904s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.482146932 (61h12m17.353229977s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.484118316 (61h12m17.355201361s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.486216912 (61h12m17.357299957s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.487427918 (61h12m17.358510953s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:19.196950317 (61h12m19.068033352s) [upload=53698]: Next returned nil segment
Dec 8 04:41:19.196972289 (61h12m19.068055324s) [upload=53698]: Waiting for error group managing segments...
Dec 8 04:41:19.470957375 (61h12m19.34204042s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.478965694 (61h12m19.350048739s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.488459723 (61h12m19.359542758s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.491834840 (61h12m19.362917885s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.496467461 (61h12m19.367550506s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.497643140 (61h12m19.368726185s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.513620774 (61h12m19.384703809s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.519169180 (61h12m19.390252225s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.582513759 (61h12m19.453596794s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.600844513 (61h12m19.471927549s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.627512986 (61h12m19.498596031s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.923827261 (61h12m19.794910306s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940676124 (61h12m19.811759159s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940696483 (61h12m19.811779518s) [seg_pos={0 0}, upload=53698]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:41:19.940920650 (61h12m19.812003695s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.940927292 (61h12m19.812010347s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941137082 (61h12m19.812220117s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941146079 (61h12m19.812229124s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941154806 (61h12m19.812237851s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941162220 (61h12m19.812245255s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941203298 (61h12m19.812286343s) [seg_pos={0 0}, upload=53698]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:41:19.941272510 (61h12m19.812355555s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:41:19.941277359 (61h12m19.812360404s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:50:23.376780170 (61h21m23.247863205s) [upload=53698]: Done uploading segments...
Dec 8 04:50:23.376921268 (61h21m23.248004313s) [upload=53698]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.833477891s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............................................X..X..........X] duration:
9m14.344512786s trace:
https://jaeger.dev.storj.io/trace/327d8af62af2ad4a
FAIL (2025-12-08 04:50:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 8 04:41:10.986572683 (61h12m10.857655728s) [upload=53698]: Starting upload
Dec 8 04:41:10.986697500 (61h12m10.857780545s) [upload=53698]: Done starting upload
Dec 8 04:41:10.986782503 (61h12m10.857865538s) [upload=53698]: Uploading segments...
Dec 8 04:41:10.987063708 (61h12m10.858146753s) [upload=53698]: Got next segment. Inline: false
Dec 8 04:41:10.987116498 (61h12m10.858199543s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 8 04:41:10.987121758 (61h12m10.858204803s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 8 04:41:17.462876699 (61h12m17.333959744s) [seg_pos={0 0}, upload=53698]: Begin upload segment...
Dec 8 04:41:17.462933336 (61h12m17.334016381s) [seg_pos={0 0}, upload=53698]: Done begin upload segment.
Dec 8 04:41:17.462963864 (61h12m17.334046909s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464241417 (61h12m17.335324452s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.464758772 (61h12m17.335841817s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465245028 (61h12m17.336328063s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.465727617 (61h12m17.336810672s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.466300738 (61h12m17.337383783s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.467904602 (61h12m17.338987647s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.469643584 (61h12m17.340726629s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.470720414 (61h12m17.341803449s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.472841653 (61h12m17.343924708s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.474783842 (61h12m17.345866877s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.478925869 (61h12m17.350008904s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.482146932 (61h12m17.353229977s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.484118316 (61h12m17.355201361s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.486216912 (61h12m17.357299957s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:17.487427918 (61h12m17.358510953s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Uploading piece...
Dec 8 04:41:19.196950317 (61h12m19.068033352s) [upload=53698]: Next returned nil segment
Dec 8 04:41:19.196972289 (61h12m19.068055324s) [upload=53698]: Waiting for error group managing segments...
Dec 8 04:41:19.470957375 (61h12m19.34204042s) [noise=true , address=145.79.227.40:4051 , piece_id=Z3JV2JYCGSZTSOKXPQFLG3LFXE3XSZXMEI6RKLWR7AIJWVE7F5VA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.478965694 (61h12m19.350048739s) [noise=true , address=145.79.227.22:4510 , piece_id=ME3CNTADT3MKZ7A7O35HQ77I3QEJ4RDHZ3HWNXTK74WCXXTVFGHA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.488459723 (61h12m19.359542758s) [noise=true , address=145.79.227.29:4943 , piece_id=W7RBOTEMANEW5K2RNN5VH25ZZWR7ENEZIWZYKGNL7CLYMFP5KSYA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.491834840 (61h12m19.362917885s) [noise=true , address=145.79.227.39:4014 , piece_id=42YQDT5VNDBCG42TGQE5XEO7V4M3FLXEBPCWNWF4VTDD4GVTD2MQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.496467461 (61h12m19.367550506s) [noise=true , address=145.79.227.28:4725 , piece_id=2KZFS66KORIJR7AJXPFK2YDIU5BGRNNYFUSMF2QCJHBXH4UJ5VLQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.497643140 (61h12m19.368726185s) [noise=true , address=145.79.227.26:4653 , piece_id=VXRBIXTQMICMWYO3QUTXC6WYEAMWLH5HDM6DGA3QSVJT6XUYQ4CQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.513620774 (61h12m19.384703809s) [noise=true , address=145.79.227.21:4474 , piece_id=ALCSXYN2JKG3WWKBYAGU3HSJGILRK324Q6BFUB3XAN7BRWIFUS6Q, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.519169180 (61h12m19.390252225s) [noise=true , address=145.79.227.24:4583 , piece_id=D66DYNF7HFKQXG2BZ7A7MS4TQQEUT7VXEUB4Q6K5E45QQ3KBLRPQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.582513759 (61h12m19.453596794s) [noise=true , address=77.90.31.18:4807 , piece_id=CG6IOIJXBGUWZHHSTI3CYHGQFHKJIUENOO7SXIMHMSKPT6KJC7OQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.600844513 (61h12m19.471927549s) [noise=true , address=77.90.31.13:4628 , piece_id=U3GVPVFRIIIOKMG2LTQZ7HLFQ4CKA3PJ7KDMTN37SRUVLXQ7PBHQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.627512986 (61h12m19.498596031s) [noise=true , address=77.90.31.10:4518 , piece_id=PVAIQII44G5U6ULAIESSKG65ZG4NH7DLS4DR2D4PRVULTRBDN2UQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.923827261 (61h12m19.794910306s) [noise=true , address=145.79.227.42:4123 , piece_id=SUGMMCANVZOSRM6KJF2JZFNGKCS76A3BIVR7FPEVGIFS3UXYSCBA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940676124 (61h12m19.811759159s) [noise=true , address=145.79.227.26:4654 , piece_id=KRO6XVQCHI3CYVUGBH4N5LLMUDTN2TTI3D2TR7F5YB24KOCIKDJQ, seg_pos={0 0}, upload=53698]: Done uploading piece. err: <nil>
Dec 8 04:41:19.940696483 (61h12m19.811779518s) [seg_pos={0 0}, upload=53698]: Segment reached optimal threshold of 13 pieces.
Dec 8 04:41:19.940920650 (61h12m19.812003695s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.940927292 (61h12m19.812010347s) [noise=true , address=145.79.227.33:4871 , piece_id=CSYTGISPEKL3YEHEE5YLAPJQJ76C7H4KR44ASGSWPHWIQCD4R7RA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941137082 (61h12m19.812220117s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941146079 (61h12m19.812229124s) [noise=true , address=145.79.227.21:4475 , piece_id=JZSLVHKHWCDZQM64QVSQYXZPORC63IY3U43X73Q5VPUDD3XCPKQA, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941154806 (61h12m19.812237851s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a9UG3xktEYLqGrk6Z1YRdzSd7EF8CpjJFLdLHi46PUXS92rSg): protocol: expected piece hash; context canceled; EOF
Dec 8 04:41:19.941162220 (61h12m19.812245255s) [noise=true , address=145.79.227.27:4691 , piece_id=ESMSMYRQCCYAKGQYZCILQNNWHV3WACMYBDPEAEE44OUGZK3OGK3A, seg_pos={0 0}, upload=53698]: Long tail cancellation due to optimal threshold
Dec 8 04:41:19.941203298 (61h12m19.812286343s) [seg_pos={0 0}, upload=53698]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 8 04:41:19.941272510 (61h12m19.812355555s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 04:41:19.941277359 (61h12m19.812360404s) [upload=53698]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 04:50:23.376780170 (61h21m23.247863205s) [upload=53698]: Done uploading segments...
Dec 8 04:50:23.376921268 (61h21m23.248004313s) [upload=53698]: Upload finished. err: metaclient: context deadline exceeded
Duration: 9m14.344536772s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..........X..................................X..............] duration:
679.438403ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:50:16 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 679.469933ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
473.366435ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:50:14 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 473.399088ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.726542886s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:50:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.726576068s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X..........................................X.............] duration:
5.776355638s trace:
https://jaeger.dev.storj.io/trace/3d9c5ba0a6d4da26
FAIL (2025-12-08 04:50:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.776436061s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [......................XX....................................] duration:
61.18117ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:50:02 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 61.223229ms
edge
linksharing
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [...X....X........X............X.X...........................] duration:
3.47110602s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:49:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.47115375s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...........X..................................X.............] duration:
706.609041ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:49:58 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 706.636041ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.128727109s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:49:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.128755863s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.839526296s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:49:37 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.839557866s
core
uplink
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.955288338s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:49:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.955369474s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
466.980895ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:49:24 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.008948ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [............X..................................X............] duration:
684.264351ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:49:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 684.291845ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.......................XX...................................] duration:
25.502602ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:49:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 25.536677ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.883395324s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:49:19 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.883425059s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....X..........................................X............] duration:
5.712317899s trace:
https://jaeger.dev.storj.io/trace/3d9c5ba0a6d4da26
FAIL (2025-12-08 04:49:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.71234971s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [....X....X........X............X.X..........................] duration:
3.510210171s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:48:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.510242411s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.811395613s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:48:52 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.811417284s
core
uplink
us1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.............X..................................X...........] duration:
676.920399ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:48:48 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 676.955855ms
core
satellite
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.672185301s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:48:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.672214837s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [........................XX..................................] duration:
47.080339ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:48:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 47.109405ms
edge
linksharing
bgp
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.839547238s trace:
https://jaeger.dev.storj.io/trace/6506fd1cc11bbd33
FAIL (2025-12-08 04:48:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.839576233s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....X..........................................X...........] duration:
5.963688946s trace:
https://jaeger.dev.storj.io/trace/3d9c5ba0a6d4da26
FAIL (2025-12-08 04:48:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.963735636s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
478.729818ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:48:11 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 478.760758ms
core
satellite
eu1
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..............X..................................X..........] duration:
2.370924946s remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:48:02 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 2.370961635s
core
satellite
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [.........................XX.................................] duration:
27.950233ms remote:
"tcp:207.211.208.129:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/2af55367efa5ea5e
FAIL (2025-12-08 04:47:53 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 27.975731ms
edge
linksharing
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.625728987s trace:
https://jaeger.dev.storj.io/trace/a7f387c7d7e5198
FAIL (2025-12-08 04:47:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.625771789s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.....X....X........X............X.X.........................] duration:
3.55480631s trace:
https://jaeger.dev.storj.io/trace/498ca12c4376e6b3
FAIL (2025-12-08 04:47:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.554826398s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......X..........................................X..........] duration:
6.799414979s trace:
https://jaeger.dev.storj.io/trace/3d9c5ba0a6d4da26
FAIL (2025-12-08 04:47:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.799472399s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...............X..................................X.........] duration:
800.22954ms remote:
"tcp:35.212.10.183:443" target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4679b6ba9d140ac6
FAIL (2025-12-08 04:47:18 UTC)
orbiter warning: Get "https://us1.storj.io/login": EOF
Duration: 800.260188ms
core
satellite
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
472.881251ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/17143a9730f83ede
FAIL (2025-12-08 04:47:14 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 472.908312ms
core
satellite
eu1
|