Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 6.925603955s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:37:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.925650724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X.XX......................] duration: 5.801579283s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:37:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.801601656s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 6.944306903s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:37:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.944334746s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X.XX.....................] duration: 6.116147452s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:37:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.116171567s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................X.XX....................] duration: 6.033279654s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:36:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.03330404s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 9.507460284s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:36:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.507490753s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X.XX...................] duration: 6.100156378s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:35:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.100181766s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 6.658813014s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:35:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.658839143s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 6.881001195s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:34:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.881015551s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................X.XX..................] duration: 6.229657101s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:34:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.22968828s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 6.648494193s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:34:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.648541063s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X.XX.................] duration: 6.304446374s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:33:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.304472714s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 6.799320373s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:33:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.799350509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................X.XX................] duration: 6.812628712s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:32:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.812655454s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................X.XX...............] duration: 5.882435834s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:32:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.882476051s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 6.642134899s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:32:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.64216274s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................X.XX..............] duration: 6.269962613s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:31:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.270012078s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 6.644088597s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:31:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.644135968s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................................X.XX.............] duration: 5.652043494s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:30:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.65211969s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 6.648524703s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:30:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.648555602s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................................X.XX............] duration: 6.36959516s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:29:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.369618275s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................X.XX...........] duration: 6.501878551s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:29:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.50189904s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 6.509169114s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:29:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.50918712s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 6.607989753s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:28:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.608006275s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................X.XX..........] duration: 6.295592894s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.295619194s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 6.782690244s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:28:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.782776398s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 6.539209187s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:27:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.539225326s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................X.XX.........] duration: 6.077272067s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:27:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.077285232s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 6.75331154s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:27:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.75333759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................X.XX........] duration: 5.604068831s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:26:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.604093919s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 6.674582114s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:26:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.674624876s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................X.XX.......] duration: 6.38220267s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:25:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.382233929s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 6.739956266s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:25:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.739980112s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X.XX......] duration: 5.288740544s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:25:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.288766904s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 6.926468545s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:25:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.926491667s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X.XX.....] duration: 5.594753731s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:25:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.594776035s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 6.831507709s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:25:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.831525743s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X.XX....] duration: 6.422632789s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:24:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.422659921s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................X.XX...] duration: 5.653861646s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:24:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.65388495s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 6.644118514s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:24:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.644133131s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 7.235271247s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:23:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.235294462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.XX..] duration: 5.770565549s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:23:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.770588483s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 6.782767601s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:22:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.782792137s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X.XX.] duration: 6.512822786s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:22:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.512836191s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................X.XX] duration: 1m0.251161023s trace: https://jaeger.dev.storj.io/trace/764d2da8c4c47d1d FAIL (2026-05-15 10:21:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== May 15 10:20:54.636559976 (351h33m5.157383721s) [upload=336815]: Starting upload May 15 10:20:54.636641471 (351h33m5.157465236s) [upload=336815]: Done starting upload May 15 10:20:54.636749597 (351h33m5.157573342s) [upload=336815]: Uploading segments... May 15 10:20:54.636907189 (351h33m5.157730934s) [upload=336815]: Got next segment. Inline: false May 15 10:20:54.636954057 (351h33m5.157777802s) [upload=336815]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:20:54.636959888 (351h33m5.157783633s) [upload=336815]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:20:54.874108411 (351h33m5.394932166s) [seg_pos={0 0}, upload=336815]: Begin upload segment... May 15 10:20:54.874180689 (351h33m5.395004424s) [seg_pos={0 0}, upload=336815]: Done begin upload segment. May 15 10:20:54.874232688 (351h33m5.395056423s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874257956 (351h33m5.395081701s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.874857841 (351h33m5.395681586s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875329101 (351h33m5.396152846s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.875789038 (351h33m5.396612783s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876122735 (351h33m5.39694647s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.876675039 (351h33m5.397498794s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877160315 (351h33m5.39798406s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.877608240 (351h33m5.398431995s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.878521873 (351h33m5.399345618s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879278507 (351h33m5.400102252s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.879573550 (351h33m5.400397295s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880622051 (351h33m5.401445796s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.880937763 (351h33m5.401761508s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.881781913 (351h33m5.402605658s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:54.885534584 (351h33m5.406358329s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Uploading piece... May 15 10:20:56.232984223 (351h33m6.753807968s) [upload=336815]: Next returned nil segment May 15 10:20:56.232994713 (351h33m6.753818458s) [upload=336815]: Waiting for error group managing segments... May 15 10:20:56.603254465 (351h33m7.12407823s) [noise=true , address=167.17.72.71:4809 , piece_id=B4WJC2R6KNSSAEBRUHA5XVNQYH2XA4UL2BVZ6AKIWJB7S5FAPVLA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.747922319 (351h33m7.268746064s) [noise=true , address=145.79.227.50:4399 , piece_id=LVBKTHZPWY32C3COR247KDLX7Y6AZ3JWYJHI6OXVSUHKQ4GQOTYA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.756871374 (351h33m7.27769512s) [noise=true , address=145.79.227.42:4133 , piece_id=NVRPGK4NXJQIGHZXTNCNLFJ346GZFI4QQTW6LZUJAQKFKZLUNN4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.766775293 (351h33m7.287599058s) [noise=true , address=145.79.227.10:4420 , piece_id=GYO2XBBPBZZVZPLNKTXBDIEORIXQI26MCBKUZJGCME2JD3NF6V6A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.771654483 (351h33m7.292478228s) [noise=true , address=145.79.227.20:4456 , piece_id=ISG3CKK25BSQ5PR676T6XE5FOPASYACHVCLYNF6SDPZNTIL2AYKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.779546172 (351h33m7.300369917s) [noise=true , address=145.79.227.16:4944 , piece_id=PWABZ6UFDTEDMPPGLJESDQAMZQH6JAIY5JAJY6ZWW7VG6PFJKECQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.808959028 (351h33m7.329782773s) [noise=true , address=145.79.227.18:4032 , piece_id=O6VH6Z33Q6MPBKAIRH2WTXA5WLOQXOPDQDMCJWRQKN7LEK3QHOKA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.814476285 (351h33m7.33530003s) [noise=true , address=145.79.227.7:4306 , piece_id=N4OUDZI7J3RHNXSFA4QULFU3JCXIL7KA462VXMLPXWMAYQ4W2H2A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.840933010 (351h33m7.361756755s) [noise=true , address=77.90.31.16:4761 , piece_id=JH2WTZPM6F7WX45VHZGBP4W3DZHUG6KO5TIRJAQ7OUCHGSPOBWZA, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:56.880471756 (351h33m7.401295501s) [noise=true , address=77.90.31.10:4532 , piece_id=7FW5C6UWBZ7GOZECOU3ALEJVLLHMK2OGTG4CAHSBYXHKYVVDP32A, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:20:57.114960213 (351h33m7.635783968s) [noise=true , address=45.91.116.10:4086 , piece_id=C4PMJ4VNRI3XHAMWG5PKPPUO3JYIR6G2J27RSGZDMWB2EITXY6WQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:29.460027942 (351h33m39.980851687s) [noise=true , address=193.149.164.79:4522 , piece_id=ITUPBPLYYX3XYZV35M4HVFMIR6RIUU5KK6Q37EYQNAVJ62X6PA4Q, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:38.526247743 (351h33m49.047071488s) [noise=true , address=193.149.164.78:4485 , piece_id=3TUFWC6E47VBZCTJ22MY4PDRUZT5ELE5O6LIS2W5N73O7FBSR7UQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: <nil> May 15 10:21:53.976029715 (351h34m4.49685346s) [noise=true , address=193.149.164.70:4140 , piece_id=7SZPUIC7AHKWU2IVZCIR332ERASFS3EGKUVRGPP2YXWIJ72VDHKQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1RGXZEGGB2pi2NjhSth78NDu6ESxqNiscoEdAaFei9SRw631UJ, address:193.149.164.70:4140): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976079140 (351h34m4.496902895s) [noise=true , address=193.149.164.67:4029 , piece_id=OUMLWRO3UDFG2YAAOE4LSNXPFILHPVRQNP6H6KHGETE54KDWXMUQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1NdCch8CW27VvNyqNPaMW1B4CEvguPswFjswRP4R16VieeFaSe, address:193.149.164.67:4029): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976255637 (351h34m4.497079382s) [noise=true , address=193.149.164.66:4996 , piece_id=MDHP2GNFSQ2YUIPHD3KFSFDJXELEGJYUWNQHZ4HDB6ICH5RLCHQQ, seg_pos={0 0}, upload=336815]: Done uploading piece. err: ecclient: upload failed (node:1sK7oayGYBqB4fhjodVcZLid7438rnseaGQqTuZ4pTU5YNSuHi, address:193.149.164.66:4996): protocol: context deadline exceeded; context deadline exceeded May 15 10:21:53.976312626 (351h34m4.497136371s) [seg_pos={0 0}, upload=336815]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> May 15 10:21:54.225022388 (351h34m4.745846133s) [upload=336815]: Done uploading segments... May 15 10:21:54.225096430 (351h34m4.745920175s) [upload=336815]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.251170601s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 6.44073531s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:21:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.440762501s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 7.102827584s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:21:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.102842113s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.000868925s trace: https://jaeger.dev.storj.io/trace/6c074da42d5e62fc FAIL (2026-05-15 10:20:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000878203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1m0.242517724s trace: https://jaeger.dev.storj.io/trace/19a737610c70e61f FAIL (2026-05-15 10:20:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 15 10:19:54.140749183 (351h32m4.661572928s) [upload=336801]: Starting upload May 15 10:19:54.140806733 (351h32m4.661630478s) [upload=336801]: Done starting upload May 15 10:19:54.140843693 (351h32m4.661667438s) [upload=336801]: Uploading segments... May 15 10:19:54.140993369 (351h32m4.661817144s) [upload=336801]: Got next segment. Inline: false May 15 10:19:54.141039376 (351h32m4.661863121s) [upload=336801]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 15 10:19:54.141045088 (351h32m4.661868843s) [upload=336801]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 15 10:19:54.384175935 (351h32m4.90499968s) [seg_pos={0 0}, upload=336801]: Begin upload segment... May 15 10:19:54.384307736 (351h32m4.905131481s) [seg_pos={0 0}, upload=336801]: Done begin upload segment. May 15 10:19:54.384354686 (351h32m4.905178431s) [noise=true , address=145.79.227.52:4463 , piece_id=U36DVAW4UBTJAH5NG7QUHOT4NR75YM4PRYWGEG2533Z5PQSJAANQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.384364805 (351h32m4.90518855s) [noise=true , address=45.91.116.8:4078 , piece_id=PBT3FKZ2LNALH4OSDUS3BBBLOSFSGTNAXDHWH6Z6XKBR6ON25YOQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.384551721 (351h32m4.905375466s) [noise=true , address=145.79.227.9:4374 , piece_id=OAZTARK6M4C4DP52L5KZHO576CGCSX6LWP4FHXHWAEOCC4LSQ6QQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.385144252 (351h32m4.905967997s) [noise=true , address=193.149.164.70:4123 , piece_id=LAXORCLDBT3C63ZYU263XSF753YIQ65TWOSPIUZ3SAQYT24GMIJA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.385367418 (351h32m4.906191163s) [noise=true , address=193.149.164.67:4021 , piece_id=FGEU3VKR6YP6XQSHDF5L3QNYVXDWMJBHZQMPFOSCZ4LRJUBK6DBQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.385619269 (351h32m4.906443014s) [noise=true , address=193.149.164.78:4490 , piece_id=75YPXV5NTFLXW4CE4E3SV6GUVKXG66YJR7JJJ2ZT2YUI7C6A2XKA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.385871440 (351h32m4.906695185s) [noise=true , address=145.79.227.21:4482 , piece_id=FIIXOVQOPUVP74FMZ2VDTOMJCPWEEOHKI3U6MBYU6TAG2OCMFOJA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.386104755 (351h32m4.9069285s) [noise=true , address=77.90.31.18:4820 , piece_id=LU4R4IKNWYN7ANIWRGJWVANJMPYXVIVWFUEMDFCEKRW5MDF4WWWA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.386342188 (351h32m4.907165933s) [noise=true , address=145.79.227.37:4577 , piece_id=K5C2VE5Y4FO2F22FJBPIBOY4JZOMZBO4USRWMFTHOMJP3OUMCHNA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.386568680 (351h32m4.907392425s) [noise=true , address=167.17.72.71:4790 , piece_id=C7BQLVSDTKAO3ZO52PVELDLG4FELPUJVHTTPBED5BAN3DAMVG4PQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.386794762 (351h32m4.907618507s) [noise=true , address=193.149.164.66:4977 , piece_id=S4WCTFTVE6PZYOJNQHPLBBWDTDRBS2F4QUOQIOJMBAI2TMARSQLQ, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.387027977 (351h32m4.907851722s) [noise=true , address=193.149.164.68:4035 , piece_id=2O4T3V7NLOSXRSR4ITXOTHG5SDZ77MIQ5CZDI7IERXARBHW6ITYA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.387251162 (351h32m4.908074908s) [noise=true , address=167.17.72.70:4747 , piece_id=IF2OTMPJ2J6XQWEHGHZEDGQ2HV4GSNYKPXABFRWT76QQWXFTMF4Q, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.387467215 (351h32m4.90829096s) [noise=true , address=193.149.164.79:4518 , piece_id=3R22EIXZD4PWYB3YUKWAGLXGSWJ5TI6LZBTPOQ4PSWWTD7QMY6LA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.388992435 (351h32m4.90981621s) [noise=true , address=145.79.227.31:4812 , piece_id=NATDSQEASDN5WK76HWQQZUHKUO4SJV7533BTQ4EKZ7KU7T7I2NXA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:54.389739160 (351h32m4.910562915s) [noise=true , address=167.17.72.69:4717 , piece_id=LR34EM6HSWFBERRQT5FQGHJ2XAHME3AEYGW7ULOLD2GYFLJGH2GA, seg_pos={0 0}, upload=336801]: Uploading piece... May 15 10:19:55.734563919 (351h32m6.255387654s) [upload=336801]: Next returned nil segment May 15 10:19:55.734579288 (351h32m6.255403033s) [upload=336801]: Waiting for error group managing segments... May 15 10:19:56.102402051 (351h32m6.623225796s) [noise=true , address=167.17.72.71:4790 , piece_id=C7BQLVSDTKAO3ZO52PVELDLG4FELPUJVHTTPBED5BAN3DAMVG4PQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.127670708 (351h32m6.648494453s) [noise=true , address=167.17.72.69:4717 , piece_id=LR34EM6HSWFBERRQT5FQGHJ2XAHME3AEYGW7ULOLD2GYFLJGH2GA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.226903643 (351h32m6.747727388s) [noise=true , address=167.17.72.70:4747 , piece_id=IF2OTMPJ2J6XQWEHGHZEDGQ2HV4GSNYKPXABFRWT76QQWXFTMF4Q, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.261839397 (351h32m6.782663142s) [noise=true , address=145.79.227.9:4374 , piece_id=OAZTARK6M4C4DP52L5KZHO576CGCSX6LWP4FHXHWAEOCC4LSQ6QQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.273759042 (351h32m6.794582787s) [noise=true , address=145.79.227.31:4812 , piece_id=NATDSQEASDN5WK76HWQQZUHKUO4SJV7533BTQ4EKZ7KU7T7I2NXA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.276132851 (351h32m6.796956596s) [noise=true , address=145.79.227.52:4463 , piece_id=U36DVAW4UBTJAH5NG7QUHOT4NR75YM4PRYWGEG2533Z5PQSJAANQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.309607878 (351h32m6.830431623s) [noise=true , address=77.90.31.18:4820 , piece_id=LU4R4IKNWYN7ANIWRGJWVANJMPYXVIVWFUEMDFCEKRW5MDF4WWWA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.322437979 (351h32m6.843261725s) [noise=true , address=145.79.227.21:4482 , piece_id=FIIXOVQOPUVP74FMZ2VDTOMJCPWEEOHKI3U6MBYU6TAG2OCMFOJA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:56.927884852 (351h32m7.448708597s) [noise=true , address=45.91.116.8:4078 , piece_id=PBT3FKZ2LNALH4OSDUS3BBBLOSFSGTNAXDHWH6Z6XKBR6ON25YOQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:19:57.222281133 (351h32m7.743104878s) [noise=true , address=145.79.227.37:4577 , piece_id=K5C2VE5Y4FO2F22FJBPIBOY4JZOMZBO4USRWMFTHOMJP3OUMCHNA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:20:05.280591500 (351h32m15.801415245s) [noise=true , address=193.149.164.79:4518 , piece_id=3R22EIXZD4PWYB3YUKWAGLXGSWJ5TI6LZBTPOQ4PSWWTD7QMY6LA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:20:06.728725433 (351h32m17.249549178s) [noise=true , address=193.149.164.78:4490 , piece_id=75YPXV5NTFLXW4CE4E3SV6GUVKXG66YJR7JJJ2ZT2YUI7C6A2XKA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: <nil> May 15 10:20:53.479623591 (351h33m4.000447346s) [noise=true , address=193.149.164.66:4977 , piece_id=S4WCTFTVE6PZYOJNQHPLBBWDTDRBS2F4QUOQIOJMBAI2TMARSQLQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: ecclient: upload failed (node:12aJKWDA6NWccf6vhQ17rys2b5yEE3wjabLiJ4rvUsA3Nc5sx4u, address:193.149.164.66:4977): protocol: rpc: dial tcp 193.149.164.66:4977: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 10:20:53.479681342 (351h33m4.000505087s) [noise=true , address=193.149.164.70:4123 , piece_id=LAXORCLDBT3C63ZYU263XSF753YIQ65TWOSPIUZ3SAQYT24GMIJA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: ecclient: upload failed (node:12qEH9s4rF7vREESy7WfXwy1bXFFtM6mTbcGZvsq3NRVWraH2xW, address:193.149.164.70:4123): protocol: rpc: dial tcp 193.149.164.70:4123: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 10:20:53.480005580 (351h33m4.000829325s) [noise=true , address=193.149.164.68:4035 , piece_id=2O4T3V7NLOSXRSR4ITXOTHG5SDZ77MIQ5CZDI7IERXARBHW6ITYA, seg_pos={0 0}, upload=336801]: Done uploading piece. err: ecclient: upload failed (node:1PYXKEnZgGki35kXwR94A3Bdz2VkNSqZeSLdKkMVWgVNJcPRfj, address:193.149.164.68:4035): protocol: rpc: dial tcp 193.149.164.68:4035: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 15 10:20:53.480269984 (351h33m4.001093729s) [noise=true , address=193.149.164.67:4021 , piece_id=FGEU3VKR6YP6XQSHDF5L3QNYVXDWMJBHZQMPFOSCZ4LRJUBK6DBQ, seg_pos={0 0}, upload=336801]: Done uploading piece. err: ecclient: upload failed (node:1cEcvJmojoQWjP6reuTApwMFTozSJXPJ9uez1S5yWmiiBqj8aN, address:193.149.164.67:4021): protocol: write tcp 188.166.225.170:43394->193.149.164.67:4021: use of closed network connection; conn closed: op May 15 10:20:53.480299250 (351h33m4.001122995s) [seg_pos={0 0}, upload=336801]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 15 10:20:53.720156912 (351h33m4.240980657s) [upload=336801]: Done uploading segments... May 15 10:20:53.720221736 (351h33m4.241045471s) [upload=336801]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.242538593s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.00144682s trace: https://jaeger.dev.storj.io/trace/35726dfda80ac4b8 FAIL (2026-05-15 10:19:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001466968s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)