Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 5.000992984s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:09:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001018532s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.437976891s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:08:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.438007879s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 3.722182865s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:08:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.722200068s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 3.490745432s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:07:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.490777964s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 3.350076061s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:06:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.35011243s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 4.713939127s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:06:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.713978732s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 3.420888322s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:05:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.420920682s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 5.00174229s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:05:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001776136s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 4.215733173s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:03:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.215763862s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.811375579s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:02:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.811401608s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 3.460837214s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:02:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.460865627s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 3.445363509s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:01:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.44539035s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.380537877s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:00:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.380589736s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.619251816s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 13:00:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.619300469s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.003547676s trace: https://jaeger.dev.storj.io/trace/6d384e42f51d8606 FAIL (2025-11-30 12:59:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.003581281s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.748916517s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:49:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.748966302s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.611426946s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:48:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.611474256s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 5.657295581s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:47:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.657327062s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 5.662175909s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:46:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.662206418s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXXXX.......................................................] duration: 5.631752901s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:45:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.631780932s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.XXXXX......................................................] duration: 5.732957085s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:45:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.733010006s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..XXXXX.....................................................] duration: 5.53016424s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:44:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.530193967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...XXXXX....................................................] duration: 5.71854427s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:43:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.718572875s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [....XXXXX...................................................] duration: 5.637457915s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:43:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.637661072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....XXXXX..................................................] duration: 5.698452309s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:42:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.698495131s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [......XXXXX.................................................] duration: 5.669902124s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:41:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.669927132s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......XXXXX................................................] duration: 5.572724024s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:40:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.572768497s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [........XXXXX...............................................] duration: 5.644058433s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:39:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.64415579s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........XXXXX..............................................] duration: 5.566305051s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:38:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.566331251s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........XXXXX.............................................] duration: 5.760631142s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:38:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.760661088s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........XXXXX............................................] duration: 5.669642188s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:37:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.669669911s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [............XXXXX...........................................] duration: 5.678325435s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:36:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.678379368s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............XXXXX..........................................] duration: 5.788311055s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:36:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.788339288s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............XXXXX.........................................] duration: 5.704563934s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:34:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.704591328s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...............XXXXX........................................] duration: 5.89171068s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:34:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.891765104s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [................XXXXX.......................................] duration: 5.683382805s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:33:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.683410077s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................XXXXX......................................] duration: 5.808309494s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:32:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.808343308s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................XXXXX.....................................] duration: 5.726554326s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:31:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.726584603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...................XXXXX....................................] duration: 5.740929047s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:30:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.740968873s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................XXXXX...................................] duration: 5.678925516s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:29:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.678960092s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....................XXXXX..................................] duration: 5.579583503s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:28:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.57961318s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................XXXXX.................................] duration: 5.734776346s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:27:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.734806724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................XXXXX................................] duration: 5.633762599s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:26:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.633796291s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [........................XXXXX...............................] duration: 6.016761448s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:26:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 6.016792997s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................XXXXX..............................] duration: 5.748795633s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:25:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.74882537s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........................XXXXX.............................] duration: 5.685007059s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:24:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.685040042s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........................XXXXX............................] duration: 5.631881838s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:23:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.631993752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [............................XXXXX...........................] duration: 5.627410733s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:22:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.627436201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [.............................XXXXX..........................] duration: 5.688498104s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:21:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.688528684s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..............................XXXXX.........................] duration: 5.522062478s trace: https://jaeger.dev.storj.io/trace/56ffc589e0c5772a FAIL (2025-11-30 10:20:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== d segment... Nov 30 10:01:24.460646521 (505h57m23.012116982s) [seg_pos={0 0}, upload=445542]: Done begin upload segment. Nov 30 10:01:24.460733988 (505h57m23.012204439s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.460751932 (505h57m23.012222393s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461422129 (505h57m23.01289258s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.461580209 (505h57m23.01305066s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462103736 (505h57m23.013574197s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462448353 (505h57m23.013918804s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.462775867 (505h57m23.014246328s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463092690 (505h57m23.014563151s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463418610 (505h57m23.014889061s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.463804515 (505h57m23.015274976s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464148560 (505h57m23.015619021s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464557329 (505h57m23.01602779s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.464848092 (505h57m23.016318553s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465132794 (505h57m23.016603255s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465426274 (505h57m23.016896735s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.465704784 (505h57m23.017175235s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470301647 (505h57m23.021772098s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:24.470730052 (505h57m23.022200504s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Uploading piece... Nov 30 10:01:25.756204017 (505h57m24.307674478s) [upload=445542]: Next returned nil segment Nov 30 10:01:25.756220048 (505h57m24.307690509s) [upload=445542]: Waiting for error group managing segments... Nov 30 10:01:26.077787818 (505h57m24.629258279s) [noise=true , address=64.49.13.26:4950 , piece_id=6TNACF4ITUDIX2A6ISYVS2P2QIQ2NYMXEGYZGVLMOT2SCPQHSF6Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.163135087 (505h57m24.714605538s) [noise=true , address=64.49.13.137:4495 , piece_id=V4TGWH5YODLMGTVWM66T7PAEGQVNDEDIBV3TJL5BZ2OGQA4K4MDA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.189162669 (505h57m24.74063313s) [noise=true , address=193.149.164.143:4698 , piece_id=YPPZUYMPMTRDBJMKLBJHNCKN7325AFJYBYLWJTJ2EA4YKVFDAVXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.357484278 (505h57m24.908954739s) [noise=true , address=145.79.227.14:4848 , piece_id=Y2WS7DW5M4WZWQR6Y5D4KSMY32AQDMDUSSFISQNIDHBGIRUNE42Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.426798566 (505h57m24.978269027s) [noise=true , address=45.91.116.10:4087 , piece_id=AU5VESS732DYDLJTIDIFNKKV622LZVQ3E66IK4IOM5YXFQ535SIQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.450260893 (505h57m25.001731354s) [noise=true , address=45.91.116.11:4140 , piece_id=NKU4O7JCIIFBL73P7TOVG2EJ5GQE7TIQ6OQJP4FM7GXN5XDCZI2A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.477387297 (505h57m25.028857748s) [noise=true , address=45.91.116.9:4268 , piece_id=IXJDEQVX3CRH4THORR5SXYUZNCJYUBWPXKMGFTMTGHBMKLP2UCYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.486235215 (505h57m25.037705676s) [noise=true , address=45.91.116.13:4182 , piece_id=T34B7I4XETMSLXRL6DGP35GH7MW4GH2H6SIBLB4OTSM6VD4YRP5A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.507726178 (505h57m25.059196639s) [noise=true , address=45.91.116.7:4036 , piece_id=MVTJGTHGP4SGOW3JO7J5OVHAV7CQMMOL6OYJT6JABV7WRQX5WGXA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.515452632 (505h57m25.066923113s) [noise=true , address=45.91.116.14:4205 , piece_id=GDHENKRVLB4GPDSX7FI3HUYI5MXIIJ27HVMPMLO422CI5UQCCW7A, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:01:26.516645543 (505h57m25.068116014s) [noise=true , address=45.91.116.4:4948 , piece_id=CTCJSYVXNTHY25QDOWOUMTQWUZL6QLKBUKSNAH34VGVGOUCPUBPQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: <nil> Nov 30 10:02:23.512289546 (505h58m22.063759997s) [noise=true , address=77.90.31.11:4564 , piece_id=VGTWIP5OAQWDIV5XON2VF3K5F2M2D3MEWTLNRGZRQYEDTYPUSPRQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9, address:77.90.31.11:4564): protocol: rpc: dial tcp 77.90.31.11:4564: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512338739 (505h58m22.0638092s) [noise=true , address=77.90.31.14:4690 , piece_id=MUCEBQP3UUWPFQBIRRJD7NVMGUG2ZJUAK2ZK5FOWABVZWRBEIVYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1WPKeCcN1uEhGjZHAJgCoJMaHxAWouGCdaU1vZwAS7kX22rxcv, address:77.90.31.14:4690): protocol: rpc: dial tcp 77.90.31.14:4690: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512474608 (505h58m22.063945069s) [noise=true , address=77.90.31.12:4600 , piece_id=ZZTKLGOQRZOGPJ5GTHCVUUJW47RS4CVC7RM2ZJRNX6ZPNFEC4EYQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512632289 (505h58m22.06410275s) [noise=true , address=77.90.31.16:4753 , piece_id=PAEPZ4JV7BA5OQ4KNGXL3S7R3B72CVYSOWCYRDMWAXGOEXTMZO7Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1MUzLDSzWMc3Eru4zGrH3yF7aGiEmfGvzfUTm9cKNagjytQ7bE, address:77.90.31.16:4753): protocol: rpc: dial tcp 77.90.31.16:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512871675 (505h58m22.064342126s) [noise=true , address=77.90.31.17:4773 , piece_id=XS3JA3ZEMXFYLQRH3YAHBYRZGW72JHWHGU2OHJVHPRZ5S2LGBXJQ, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1mpRyPuesFpUqZvgaXpQ2HDm8ccFRBJiw1QBfSN9WrRm8QSKdZ, address:77.90.31.17:4773): protocol: rpc: dial tcp 77.90.31.17:4773: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.512878688 (505h58m22.064349149s) [noise=true , address=77.90.31.10:4534 , piece_id=TKCPHE3RDNQ62M45UOIRENMUQA5UZOVA32A6K3KHJ5FAX5ZUTMQA, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:1266qFn3AGEr59ZmjnK4qg5q1s2iWD4UMTCR7tqsWP4rWfC8xWr, address:77.90.31.10:4534): protocol: rpc: dial tcp 77.90.31.10:4534: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513010088 (505h58m22.064480539s) [noise=true , address=77.90.31.21:4902 , piece_id=QW7C6PLHHINV475F4Q2Z2MIEGNHTY6WSPL4O7DWMT2AX3AR3NJ5Q, seg_pos={0 0}, upload=445542]: Done uploading piece. err: ecclient: upload failed (node:12sD3HFEaYLm7KV6uRvjbojtU9eQGMSy3Aewr4pX5Bxjy4jMCHx, address:77.90.31.21:4902): protocol: write tcp 188.166.225.170:48248->77.90.31.21:4902: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 30 10:02:23.513044093 (505h58m22.064514544s) [seg_pos={0 0}, upload=445542]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Nov 30 10:02:23.755707764 (505h58m22.307178226s) [upload=445542]: Done uploading segments... Nov 30 10:02:23.755752459 (505h58m22.30722292s) [upload=445542]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 5.522087927s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)