Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [...X....X..........X.X..X..............X....................] duration:
24.54945925s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:24:45 UTC)
orbiter warning: context deadline exceeded
Duration: 24.549494617s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
7.052432413s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:24:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.052480494s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
24 out of 60 is failed (threshold=3) [.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX.......XX] duration:
30.001431037s trace:
https://jaeger.dev.storj.io/trace/26f4eaf662f31867
FAIL (2025-12-17 15:24:36 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001445444s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [X.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXXXX..X] duration:
5.0011564s trace:
https://jaeger.dev.storj.io/trace/56c6705dc31c16ef
FAIL (2025-12-17 15:24:36 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001180967s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
4.844363288s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:24:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.844463509s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
24 out of 60 is failed (threshold=3) [X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX.......X] duration:
30.001415317s trace:
https://jaeger.dev.storj.io/trace/59abf969f94ec5fd
FAIL (2025-12-17 15:24:06 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001441748s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [XX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXXXX..] duration:
4.717175582s trace:
https://jaeger.dev.storj.io/trace/3d3fc28162ed0f93
FAIL (2025-12-17 15:23:58 UTC)
orbiter warning: context deadline exceeded
Duration: 4.717202652s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.979426345s trace:
https://jaeger.dev.storj.io/trace/36b7416309fb65ae
FAIL (2025-12-17 15:23:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 17 15:22:06.327044445 (287h53m6.1981275s) [upload=252388]: Starting upload
Dec 17 15:22:06.327150266 (287h53m6.198233311s) [upload=252388]: Done starting upload
Dec 17 15:22:06.327211142 (287h53m6.198294178s) [upload=252388]: Uploading segments...
Dec 17 15:22:06.327502838 (287h53m6.198585893s) [upload=252388]: Got next segment. Inline: false
Dec 17 15:22:06.327557081 (287h53m6.198640126s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 17 15:22:06.327563974 (287h53m6.198647019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 17 15:22:06.584040853 (287h53m6.455123908s) [seg_pos={0 0}, upload=252388]: Begin upload segment...
Dec 17 15:22:06.584114093 (287h53m6.455197138s) [seg_pos={0 0}, upload=252388]: Done begin upload segment.
Dec 17 15:22:06.584156914 (287h53m6.455239959s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584166432 (287h53m6.455249477s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584737319 (287h53m6.455820364s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584912192 (287h53m6.455995227s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585408698 (287h53m6.456491743s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585979034 (287h53m6.457062079s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586067813 (287h53m6.457150858s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586515706 (287h53m6.457598751s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586571211 (287h53m6.457654256s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586965191 (287h53m6.458048246s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587145996 (287h53m6.458229041s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587469843 (287h53m6.458552888s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587862540 (287h53m6.458945575s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588270868 (287h53m6.459353913s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588706717 (287h53m6.459789752s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.589144702 (287h53m6.460227747s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:08.286422858 (287h53m8.157505903s) [upload=252388]: Next returned nil segment
Dec 17 15:22:08.286451883 (287h53m8.157534928s) [upload=252388]: Waiting for error group managing segments...
Dec 17 15:22:08.570748085 (287h53m8.44183113s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.571608012 (287h53m8.442691057s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.580464307 (287h53m8.451547342s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.592232576 (287h53m8.463315621s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.601269753 (287h53m8.472352798s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.648702104 (287h53m8.519785149s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.664793244 (287h53m8.535876289s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.808436035 (287h53m8.67951908s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.819113357 (287h53m8.690196422s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.847682438 (287h53m8.718765473s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.254002655 (287h53m9.125085701s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.257023288 (287h53m9.128106333s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446348347 (287h53m9.317431392s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446368445 (287h53m9.31745148s) [seg_pos={0 0}, upload=252388]: Segment reached optimal threshold of 13 pieces.
Dec 17 15:22:09.446552647 (287h53m9.317635692s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446558988 (287h53m9.317642024s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446668567 (287h53m9.317751612s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446678736 (287h53m9.317761781s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446729934 (287h53m9.317812989s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446739963 (287h53m9.317823008s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446764380 (287h53m9.317847415s) [seg_pos={0 0}, upload=252388]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 17 15:22:09.446834984 (287h53m9.317918019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 17 15:22:09.446839673 (287h53m9.317922718s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 17 15:22:09.761556551 (287h53m9.632639596s) [upload=252388]: Done uploading segments...
Dec 17 15:22:09.761576910 (287h53m9.632659955s) [upload=252388]: Upload finished. err: <nil>
Duration: 5.979458046s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [....X....X..........X.X..X..............X...................] duration:
18.360804806s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:23:46 UTC)
orbiter warning: context deadline exceeded
Duration: 18.360834292s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
23 out of 60 is failed (threshold=3) [.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX.......] duration:
27.433118581s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:23:34 UTC)
orbiter warning: context deadline exceeded
Duration: 27.433153488s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXXXX.] duration:
3.386868745s trace:
https://jaeger.dev.storj.io/trace/3d3fc28162ed0f93
FAIL (2025-12-17 15:23:18 UTC)
orbiter warning: context deadline exceeded
Duration: 3.386896718s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.....X....X..........X.X..X..............X..................] duration:
15.49542645s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:23:13 UTC)
orbiter warning: context deadline exceeded
Duration: 15.495487195s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
24 out of 60 is failed (threshold=3) [X.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX......] duration:
24.22722223s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:23:00 UTC)
orbiter warning: context deadline exceeded
Duration: 24.227245694s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
4.965216105s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:22:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.965247284s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
6.286603498s trace:
https://jaeger.dev.storj.io/trace/36b7416309fb65ae
FAIL (2025-12-17 15:22:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 17 15:22:06.327044445 (287h53m6.1981275s) [upload=252388]: Starting upload
Dec 17 15:22:06.327150266 (287h53m6.198233311s) [upload=252388]: Done starting upload
Dec 17 15:22:06.327211142 (287h53m6.198294178s) [upload=252388]: Uploading segments...
Dec 17 15:22:06.327502838 (287h53m6.198585893s) [upload=252388]: Got next segment. Inline: false
Dec 17 15:22:06.327557081 (287h53m6.198640126s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 17 15:22:06.327563974 (287h53m6.198647019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 17 15:22:06.584040853 (287h53m6.455123908s) [seg_pos={0 0}, upload=252388]: Begin upload segment...
Dec 17 15:22:06.584114093 (287h53m6.455197138s) [seg_pos={0 0}, upload=252388]: Done begin upload segment.
Dec 17 15:22:06.584156914 (287h53m6.455239959s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584166432 (287h53m6.455249477s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584737319 (287h53m6.455820364s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584912192 (287h53m6.455995227s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585408698 (287h53m6.456491743s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585979034 (287h53m6.457062079s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586067813 (287h53m6.457150858s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586515706 (287h53m6.457598751s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586571211 (287h53m6.457654256s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586965191 (287h53m6.458048246s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587145996 (287h53m6.458229041s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587469843 (287h53m6.458552888s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587862540 (287h53m6.458945575s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588270868 (287h53m6.459353913s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588706717 (287h53m6.459789752s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.589144702 (287h53m6.460227747s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:08.286422858 (287h53m8.157505903s) [upload=252388]: Next returned nil segment
Dec 17 15:22:08.286451883 (287h53m8.157534928s) [upload=252388]: Waiting for error group managing segments...
Dec 17 15:22:08.570748085 (287h53m8.44183113s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.571608012 (287h53m8.442691057s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.580464307 (287h53m8.451547342s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.592232576 (287h53m8.463315621s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.601269753 (287h53m8.472352798s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.648702104 (287h53m8.519785149s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.664793244 (287h53m8.535876289s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.808436035 (287h53m8.67951908s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.819113357 (287h53m8.690196422s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.847682438 (287h53m8.718765473s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.254002655 (287h53m9.125085701s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.257023288 (287h53m9.128106333s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446348347 (287h53m9.317431392s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446368445 (287h53m9.31745148s) [seg_pos={0 0}, upload=252388]: Segment reached optimal threshold of 13 pieces.
Dec 17 15:22:09.446552647 (287h53m9.317635692s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446558988 (287h53m9.317642024s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446668567 (287h53m9.317751612s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446678736 (287h53m9.317761781s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446729934 (287h53m9.317812989s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446739963 (287h53m9.317823008s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446764380 (287h53m9.317847415s) [seg_pos={0 0}, upload=252388]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 17 15:22:09.446834984 (287h53m9.317918019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 17 15:22:09.446839673 (287h53m9.317922718s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 17 15:22:09.761556551 (287h53m9.632639596s) [upload=252388]: Done uploading segments...
Dec 17 15:22:09.761576910 (287h53m9.632659955s) [upload=252388]: Upload finished. err: <nil>
Duration: 6.286618817s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.005149884s trace:
https://jaeger.dev.storj.io/trace/36b7416309fb65ae
FAIL (2025-12-17 15:22:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Dec 17 15:22:06.327044445 (287h53m6.1981275s) [upload=252388]: Starting upload
Dec 17 15:22:06.327150266 (287h53m6.198233311s) [upload=252388]: Done starting upload
Dec 17 15:22:06.327211142 (287h53m6.198294178s) [upload=252388]: Uploading segments...
Dec 17 15:22:06.327502838 (287h53m6.198585893s) [upload=252388]: Got next segment. Inline: false
Dec 17 15:22:06.327557081 (287h53m6.198640126s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 17 15:22:06.327563974 (287h53m6.198647019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 17 15:22:06.584040853 (287h53m6.455123908s) [seg_pos={0 0}, upload=252388]: Begin upload segment...
Dec 17 15:22:06.584114093 (287h53m6.455197138s) [seg_pos={0 0}, upload=252388]: Done begin upload segment.
Dec 17 15:22:06.584156914 (287h53m6.455239959s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584166432 (287h53m6.455249477s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584737319 (287h53m6.455820364s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.584912192 (287h53m6.455995227s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585408698 (287h53m6.456491743s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.585979034 (287h53m6.457062079s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586067813 (287h53m6.457150858s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586515706 (287h53m6.457598751s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586571211 (287h53m6.457654256s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.586965191 (287h53m6.458048246s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587145996 (287h53m6.458229041s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587469843 (287h53m6.458552888s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.587862540 (287h53m6.458945575s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588270868 (287h53m6.459353913s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.588706717 (287h53m6.459789752s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:06.589144702 (287h53m6.460227747s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Uploading piece...
Dec 17 15:22:08.286422858 (287h53m8.157505903s) [upload=252388]: Next returned nil segment
Dec 17 15:22:08.286451883 (287h53m8.157534928s) [upload=252388]: Waiting for error group managing segments...
Dec 17 15:22:08.570748085 (287h53m8.44183113s) [noise=true , address=145.79.227.34:4905 , piece_id=BA4XW2UEAB2AEH732YRMXIJZ3K53XT25JEL2OOCDGJQUQQIHKSIA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.571608012 (287h53m8.442691057s) [noise=true , address=145.79.227.13:4802 , piece_id=GDMAITZ4VEGWPO33L4RDM54CCAS3XYNKWF5M2FHCZX42IVYF4LZA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.580464307 (287h53m8.451547342s) [noise=true , address=145.79.227.50:4373 , piece_id=D6NQAAWR322YXKHLDGFDBWM7LS5XGOTJG62NGWUGPU25MXACLYJQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.592232576 (287h53m8.463315621s) [noise=true , address=145.79.227.42:4121 , piece_id=N7FYKZUCIK3246MSREVXPNGXEWHKWVRFCTEAME3R7TOK7Q377AIQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.601269753 (287h53m8.472352798s) [noise=true , address=145.79.227.49:4337 , piece_id=WJZN3PR7CYBNVRKSEDXO5ZJMWTCPMA46CKKZ6VRABZ2BZ5XTT2VA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.648702104 (287h53m8.519785149s) [noise=true , address=145.79.227.45:4231 , piece_id=NCWR5TAEWA6OA4X3QRFOVTRWOKFLXXHIQUQ6SJSUY67ZIRUK3GGA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.664793244 (287h53m8.535876289s) [noise=true , address=145.79.227.33:4869 , piece_id=STVXA2MEKU4JBY6JHXEP5VIGGLWGRY4AG3VHFCEPOD2AYYOXCOMQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.808436035 (287h53m8.67951908s) [noise=true , address=77.90.31.19:4842 , piece_id=MCGM2ACRUY3KEW44OY5JP5UG7UPFBONLOLBE75FLETJGWDBSZE4A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.819113357 (287h53m8.690196422s) [noise=true , address=45.91.116.9:4263 , piece_id=CEELHO5OYUUYP3FHBFJH6YANWL3QAH6YFGNJ2ONQ3WDYD3IRAEUA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:08.847682438 (287h53m8.718765473s) [noise=true , address=45.91.116.7:4011 , piece_id=74IUNEADAD3DH3WF65OOBSSLFQA6BNZJ75G4XU6UYCNSVMUXGGDQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.254002655 (287h53m9.125085701s) [noise=true , address=45.91.116.5:4298 , piece_id=6JK6L325LV65PQPZCLHJ37VZNLZBLAKZJGNLDN7IGU6YXHYKKS3A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.257023288 (287h53m9.128106333s) [noise=true , address=45.91.116.10:4081 , piece_id=D4Q4FRQ3XGZVEIIBEV4OSDFGHFVQLJ42D24QWX7CR5A3SLLGLREA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446348347 (287h53m9.317431392s) [noise=true , address=77.90.31.17:4773 , piece_id=SDSZQ2ZBG2MSZA5IVL4IBUD3KBDELNDYWDV5FCUO5XIKIYKII3UQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: <nil>
Dec 17 15:22:09.446368445 (287h53m9.31745148s) [seg_pos={0 0}, upload=252388]: Segment reached optimal threshold of 13 pieces.
Dec 17 15:22:09.446552647 (287h53m9.317635692s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446558988 (287h53m9.317642024s) [noise=true , address=145.79.227.30:4761 , piece_id=KHD2QAVBAPMZXRGBGWNBW5EF6FPICTYSJUCKQ7XXT6I2QFB4MLGQ, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446668567 (287h53m9.317751612s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12kta1F6RNDP9GJdkiUG3x1Nk6juWJ2tmSDr599DYomSq9C1BBh): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446678736 (287h53m9.317761781s) [noise=true , address=145.79.227.44:4194 , piece_id=3O5INBADRRTTY64EGWST335PO5OGJBQGTTXNCVGH64PB6OXJEAKA, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446729934 (287h53m9.317812989s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF
Dec 17 15:22:09.446739963 (287h53m9.317823008s) [noise=true , address=45.91.116.11:4119 , piece_id=WJW5RJTQGKXQCYROLH7PRA5UWIKF55O2OP4R56IXV45B47QQ622A, seg_pos={0 0}, upload=252388]: Long tail cancellation due to optimal threshold
Dec 17 15:22:09.446764380 (287h53m9.317847415s) [seg_pos={0 0}, upload=252388]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 17 15:22:09.446834984 (287h53m9.317918019s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 17 15:22:09.446839673 (287h53m9.317922718s) [upload=252388]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 17 15:22:09.761556551 (287h53m9.632639596s) [upload=252388]: Done uploading segments...
Dec 17 15:22:09.761576910 (287h53m9.632659955s) [upload=252388]: Upload finished. err: <nil>
Duration: 30.005171185s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [......X....X..........X.X..X..............X.................] duration:
20.776839202s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:22:32 UTC)
orbiter warning: context deadline exceeded
Duration: 20.776870261s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
47 out of 60 is failed (threshold=6) [XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXXXX] duration:
5.00189534s trace:
https://jaeger.dev.storj.io/trace/3d3fc28162ed0f93
FAIL (2025-12-17 15:22:28 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001924825s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.207290138s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:22:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.207321646s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [.XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXXX] duration:
5.002154815s trace:
https://jaeger.dev.storj.io/trace/5054182428db2038
FAIL (2025-12-17 15:22:04 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002181084s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX.....] duration:
22.257860029s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:22:02 UTC)
orbiter warning: context deadline exceeded
Duration: 22.257891808s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XXX] duration:
5.001525674s trace:
https://jaeger.dev.storj.io/trace/6a0120aae2197f74
FAIL (2025-12-17 15:21:48 UTC)
orbiter warning: context deadline exceeded
Duration: 5.00154953s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.......X....X..........X.X..X..............X................] duration:
21.745439288s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:21:34 UTC)
orbiter warning: context deadline exceeded
Duration: 21.74546616s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [X..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.XX] duration:
5.00146377s trace:
https://jaeger.dev.storj.io/trace/437d0413b7c5699
FAIL (2025-12-17 15:21:17 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001487784s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
25 out of 60 is failed (threshold=3) [.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX....] duration:
15.017046043s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:21:14 UTC)
orbiter warning: context deadline exceeded
Duration: 15.01706489s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.819996646s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:21:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.820031631s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [........X....X..........X.X..X..............X...............] duration:
14.590816578s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:20:41 UTC)
orbiter warning: context deadline exceeded
Duration: 14.590846636s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [XX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.X] duration:
5.000716677s trace:
https://jaeger.dev.storj.io/trace/864dc48f305ab0a
FAIL (2025-12-17 15:20:28 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000739751s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [X.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX...] duration:
21.878700866s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:20:11 UTC)
orbiter warning: context deadline exceeded
Duration: 21.878738899s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.435904933s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:20:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.43594445s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.........X....X..........X.X..X..............X..............] duration:
14.193833792s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:20:01 UTC)
orbiter warning: context deadline exceeded
Duration: 14.193855915s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
6.520850426s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:19:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.5208788s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [.X.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX..] duration:
25.366188471s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:19:43 UTC)
orbiter warning: context deadline exceeded
Duration: 25.366269456s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
45 out of 60 is failed (threshold=6) [XXX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX.] duration:
3.561612378s trace:
https://jaeger.dev.storj.io/trace/79ff20dab7961467
FAIL (2025-12-17 15:19:38 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.561630443s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [..........X....X..........X.X..X..............X.............] duration:
16.751531296s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:19:37 UTC)
orbiter warning: context deadline exceeded
Duration: 16.75155917s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXXX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXXX] duration:
5.000984604s trace:
https://jaeger.dev.storj.io/trace/79ff20dab7961467
FAIL (2025-12-17 15:18:58 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001006445s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [...........X....X..........X.X..X..............X............] duration:
10.719934848s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:18:47 UTC)
orbiter warning: context deadline exceeded
Duration: 10.719956359s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
9.094236053s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:18:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.094264836s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [............X....X..........X.X..X..............X...........] duration:
36.450171393s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:18:36 UTC)
orbiter warning: context deadline exceeded
Duration: 36.450200949s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
27 out of 60 is failed (threshold=3) [X.X.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX.] duration:
15.50286517s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:18:32 UTC)
orbiter warning: context deadline exceeded
Duration: 15.502899565s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXXXX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXXX] duration:
5.000687328s trace:
https://jaeger.dev.storj.io/trace/4760c9f57d6e3f8d
FAIL (2025-12-17 15:17:47 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000712085s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.............X....X..........X.X..X..............X..........] duration:
17.265224268s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:17:39 UTC)
orbiter warning: context deadline exceeded
Duration: 17.265263033s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
27 out of 60 is failed (threshold=3) [.X.X.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..XX] duration:
30.000767845s trace:
https://jaeger.dev.storj.io/trace/ffa7d517ccb57b8
FAIL (2025-12-17 15:17:36 UTC)
orbiter warning: context deadline exceeded
Duration: 30.00079175s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.444052097s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:17:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.444137229s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [..............X....X..........X.X..X..............X.........] duration:
23.446833171s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:17:15 UTC)
orbiter warning: context deadline exceeded
Duration: 23.446888658s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXXXXX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXXX] duration:
5.001534293s trace:
https://jaeger.dev.storj.io/trace/19ae6be5b0708e8e
FAIL (2025-12-17 15:17:03 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001564341s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
6.326079477s trace:
https://jaeger.dev.storj.io/trace/efe666d0c62062e
FAIL (2025-12-17 15:16:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.326115475s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
26 out of 60 is failed (threshold=3) [..X.X.XX.X.X..X.XX.XX....X..X.X.X..X.XX..XXX...X..X..X..X..X] duration:
30.001776126s trace:
https://jaeger.dev.storj.io/trace/6d34dcd3937f61e0
FAIL (2025-12-17 15:16:36 UTC)
orbiter warning: context deadline exceeded
Duration: 30.001801735s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
46 out of 60 is failed (threshold=6) [XXXXXXX..XXXX.XXX.XX.X...XXX.XXX.XXXXXXXXXXXXX..XXXX.XXX.XXX] duration:
5.001465525s trace:
https://jaeger.dev.storj.io/trace/11593a1391415239
FAIL (2025-12-17 15:16:06 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001477648s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [...............X....X..........X.X..X..............X........] duration:
17.300908354s trace:
https://jaeger.dev.storj.io/trace/7a19f8e6db075595
FAIL (2025-12-17 15:16:03 UTC)
orbiter warning: context deadline exceeded
Duration: 17.300936839s
core
uplink
us-select-1
us1-placement-20
|