Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................................X..........X.........] duration: 3.904668239s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:28:50 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.904685502s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................................X..........X........] duration: 3.81045494s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:27:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.810478726s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................................X..........X.......] duration: 3.989979461s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:27:08 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.990003798s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................X..........X......] duration: 3.813049823s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:26:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.813075583s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................................X..........X.....] duration: 3.939129828s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:26:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.939154847s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............................................X..........X....] duration: 3.80723678s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:25:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.807272498s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............................................X..........X...] duration: 3.831630398s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:24:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.831695102s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............................................X..........X..] duration: 3.817402468s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:23:39 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.817430801s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................................X..........X.] duration: 3.897105949s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:22:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.897124863s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................................X..........X] duration: 5.000734503s trace: https://jaeger.dev.storj.io/trace/750702fbd4e57d58 FAIL (2026-05-15 11:22:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000752087s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 3.998463497s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:20:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.998488776s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 3.954550974s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:20:24 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.954576042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 3.831482674s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:19:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.83150621s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 3.654327838s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:18:36 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.654364077s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 3.749019989s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:17:51 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.749039225s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.657432662s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:16:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.657467748s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 3.67467368s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:16:16 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.674701494s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 3.599348375s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:15:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.599367822s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.683517246s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:15:17 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.683539018s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.860101064s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:14:07 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.860125721s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001081283s trace: https://jaeger.dev.storj.io/trace/8f32997faabe7f6 FAIL (2026-05-15 11:12:55 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.001103156s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.668520773s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:05:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.668543166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.53189018s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:04:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.531923112s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.891913109s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 11:03:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.891938206s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.179924088s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:03:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.17995191s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.XX........................................................] duration: 5.852087364s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:02:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.852114024s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.822283899s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 11:02:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.822307654s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.XX.......................................................] duration: 6.747359517s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:02:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.747385065s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.XX......................................................] duration: 5.787928746s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:01:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.787953044s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.597579134s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 11:01:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.597846153s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.XX.....................................................] duration: 6.352271458s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:01:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.352301677s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 6.840009396s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 11:00:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.840031539s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.XX....................................................] duration: 5.840609452s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 11:00:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.840635252s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 6.584347177s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:59:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.584393024s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.XX...................................................] duration: 5.751665473s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:59:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.751697734s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.XX..................................................] duration: 5.860738996s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:58:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.860762941s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 6.70064644s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:58:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.70066791s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.XX.................................................] duration: 5.945715839s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:58:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.9457424s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.XX................................................] duration: 5.926706404s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:57:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.926744087s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 7.868834256s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:57:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.868859143s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.XX...............................................] duration: 6.3774959s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:56:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.377509716s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 6.799537706s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:56:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.79956154s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.830501862s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:56:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.830524484s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.XX..............................................] duration: 6.520353384s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:56:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.520384573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.XX.............................................] duration: 5.603069048s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:55:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.603083304s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 6.685510241s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:55:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.685539016s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.XX............................................] duration: 6.095110314s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:55:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.095147766s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 7.007700085s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:54:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.00773464s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.XX...........................................] duration: 6.128985286s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:54:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.129021256s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 11.127499315s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:53:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 11.127522198s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)