Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 6.210716694s FAIL (2025-10-07 03:41:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.210754115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 6.980732351s FAIL (2025-10-07 03:41:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.980751788s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 6.143813408s FAIL (2025-10-07 03:40:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.143989473s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 5.945870852s FAIL (2025-10-07 03:39:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.945908022s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 6.182158771s FAIL (2025-10-07 03:39:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.182190842s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 6.152731198s FAIL (2025-10-07 03:38:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.152761658s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 6.088189171s FAIL (2025-10-07 03:37:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.088212814s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 6.118437657s FAIL (2025-10-07 03:37:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.118472493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 5.945303145s FAIL (2025-10-07 03:36:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.945337531s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 6.028976601s FAIL (2025-10-07 03:35:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.029009312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 6.00038203s FAIL (2025-10-07 03:34:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.000438457s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 6.015191792s FAIL (2025-10-07 03:33:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.015228522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 6.22842414s FAIL (2025-10-07 03:32:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.22845552s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 6.27445616s FAIL (2025-10-07 03:32:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.274488472s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 6.132662174s FAIL (2025-10-07 03:31:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.132699315s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 6.096287992s FAIL (2025-10-07 03:30:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.096319712s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 5.955371185s FAIL (2025-10-07 03:29:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.955410451s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 5.981084548s FAIL (2025-10-07 03:29:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.98112716s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 6.200175691s FAIL (2025-10-07 03:28:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.200244762s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 5.937137891s FAIL (2025-10-07 03:27:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.937173158s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 6.115674445s FAIL (2025-10-07 03:27:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.115709402s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.997561351s FAIL (2025-10-07 03:25:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.997592369s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 6.083348137s FAIL (2025-10-07 03:25:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.083379385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 6.083172438s FAIL (2025-10-07 03:24:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.083206902s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 6.004232604s FAIL (2025-10-07 03:24:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.004256758s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 6.051556246s FAIL (2025-10-07 03:23:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.051592544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 6.39951329s FAIL (2025-10-07 03:23:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.399548417s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 6.258717883s FAIL (2025-10-07 03:22:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.258751717s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 6.038804885s FAIL (2025-10-07 03:21:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.038836226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 5.99774787s FAIL (2025-10-07 03:20:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.997781083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 6.131525682s FAIL (2025-10-07 03:20:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.131558356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 6.308173527s FAIL (2025-10-07 03:19:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.308208834s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 6.140843548s FAIL (2025-10-07 03:18:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.140876089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 6.064580466s FAIL (2025-10-07 03:17:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.064610554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 5.926940667s FAIL (2025-10-07 03:16:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.926993688s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 6.110961375s FAIL (2025-10-07 03:16:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.111003274s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 6.166665133s FAIL (2025-10-07 03:15:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.166699979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 6.231432158s FAIL (2025-10-07 03:14:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.231468757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 6.222873482s FAIL (2025-10-07 03:14:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.222912537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 6.112043155s FAIL (2025-10-07 03:13:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.112066948s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 6.249877078s FAIL (2025-10-07 03:12:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.249913707s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 5.853618507s FAIL (2025-10-07 03:11:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.853648664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 6.060152393s FAIL (2025-10-07 03:11:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.060182952s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 6.092224367s FAIL (2025-10-07 03:10:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.092271196s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.249432632s FAIL (2025-10-07 03:09:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.249453101s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.254616884s FAIL (2025-10-07 03:08:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Oct 7 03:08:26.836660562 (325h37m12.298753035s) [noise=true , address=145.79.227.11:4731 , piece_id=NDM2HDHTCRYPFAG3WMSNXY5MI35AAEPCCKAZE2MY4Y3X2TG24APQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.836720575 (325h37m12.298813058s) [noise=true , address=145.79.227.6:4258 , piece_id=PVDGPHPSYORNDP4DJCMWJLXDM6ZLIO6VPKRY22ONYYOUZCJ5MKMA, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.837233152 (325h37m12.299325625s) [noise=true , address=45.91.116.10:4081 , piece_id=VCIGN7BIDPKG5JPPKYKBI6TPQOCRDHJYGUGU5BD7XS3HMOVQPYDQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.837488798 (325h37m12.299581271s) [noise=true , address=145.79.227.13:4804 , piece_id=NW5CQES6B7MHKX5TJ37RD66HJTOAUNI6BPJPLHZEZW3BNNU47TJA, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.837624186 (325h37m12.299716659s) [noise=true , address=77.90.31.17:4771 , piece_id=BNE2PFXM3VUNVTUGXF6AESVQRHEK5POF3JBQA52ER4RNJJYH5KAQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.838013178 (325h37m12.300105661s) [noise=true , address=77.90.31.12:4589 , piece_id=3TRWW4YAQFXIQSJ6NEN3EH7XLR45R5SOJSFWEOUGX23DCRVGG2MQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.838092649 (325h37m12.300185132s) [noise=true , address=45.91.116.11:4119 , piece_id=MZSJQMAZXHM5NZ4VEMTNYE6LODHL6VJ4FOQJKXRPWHX7KNNPDH4Q, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.838381729 (325h37m12.300474212s) [noise=true , address=145.79.227.18:4005 , piece_id=RJ7GL3RN6VKTBMTPVD7MHHCS56LPUMVA2RUYY2DYWXGNNCKXMDPA, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.838630753 (325h37m12.300723236s) [noise=true , address=45.91.116.6:4975 , piece_id=QUAC3RGAXSFZMU7LNV4SKJXV4RWKX6JQTAZOJZXTLNARMPES47QQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.838823931 (325h37m12.300916414s) [noise=true , address=77.90.31.10:4520 , piece_id=XJDCFHUMUOG6WSPE2WXHEOLAMKO2E2JFBAEPCA5BUNURGJ4W6ACQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.839175581 (325h37m12.301268244s) [noise=true , address=45.91.116.5:4297 , piece_id=RNMT5RQGCBHOBSG4BPON6VLKOO2K47E7LRXTTVGQ5VR26OA7KWAA, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.839293034 (325h37m12.301385517s) [noise=true , address=145.79.227.15:4873 , piece_id=SPQBSAW2X5OPFMRHGU4UTJQI5TY2KGGKOMK3VAWL4TZDCS7HZA7A, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.839457848 (325h37m12.301550331s) [noise=true , address=145.79.227.19:4078 , piece_id=RZ3K6YW2JCWAZKLRAYUTOAH2NKLJIZVFPRNF2M5GZ2QNRODS3P2Q, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.839774631 (325h37m12.301867114s) [noise=true , address=77.90.31.21:4879 , piece_id=RVMYYGWF5J2P7MDPEU5LSYRPWLAKEMDKQMWFJAIH4V2J2LJ46QNQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.839833854 (325h37m12.301926347s) [noise=true , address=77.90.31.13:4626 , piece_id=YGAWKEY6TTP2POSJYNALC5OVF3SKZTVW5M4FLPUNVS3MM4KZC4HQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:26.840071296 (325h37m12.302163779s) [noise=true , address=145.79.227.16:4911 , piece_id=YGGGPFLOBKEQ5UCDWCCMFPBXBHQZ3PWN2GJALMAC7KBKGIUZILTQ, seg_pos={0 0}, upload=286544]: Uploading piece... Oct 7 03:08:28.597400619 (325h37m14.059493103s) [upload=286544]: Next returned nil segment Oct 7 03:08:28.597421560 (325h37m14.059514043s) [upload=286544]: Waiting for error group managing segments... Oct 7 03:08:28.942309709 (325h37m14.404402202s) [noise=true , address=45.91.116.11:4119 , piece_id=MZSJQMAZXHM5NZ4VEMTNYE6LODHL6VJ4FOQJKXRPWHX7KNNPDH4Q, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.948620426 (325h37m14.410712899s) [noise=true , address=45.91.116.5:4297 , piece_id=RNMT5RQGCBHOBSG4BPON6VLKOO2K47E7LRXTTVGQ5VR26OA7KWAA, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.967010354 (325h37m14.429102867s) [noise=true , address=45.91.116.6:4975 , piece_id=QUAC3RGAXSFZMU7LNV4SKJXV4RWKX6JQTAZOJZXTLNARMPES47QQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.967064005 (325h37m14.429156488s) [noise=true , address=77.90.31.12:4589 , piece_id=3TRWW4YAQFXIQSJ6NEN3EH7XLR45R5SOJSFWEOUGX23DCRVGG2MQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.976494793 (325h37m14.438587276s) [noise=true , address=77.90.31.10:4520 , piece_id=XJDCFHUMUOG6WSPE2WXHEOLAMKO2E2JFBAEPCA5BUNURGJ4W6ACQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.981529129 (325h37m14.443621612s) [noise=true , address=77.90.31.17:4771 , piece_id=BNE2PFXM3VUNVTUGXF6AESVQRHEK5POF3JBQA52ER4RNJJYH5KAQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.997186203 (325h37m14.459278686s) [noise=true , address=77.90.31.13:4626 , piece_id=YGAWKEY6TTP2POSJYNALC5OVF3SKZTVW5M4FLPUNVS3MM4KZC4HQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:28.998874647 (325h37m14.46096713s) [noise=true , address=77.90.31.21:4879 , piece_id=RVMYYGWF5J2P7MDPEU5LSYRPWLAKEMDKQMWFJAIH4V2J2LJ46QNQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:29.001543900 (325h37m14.463636393s) [noise=true , address=45.91.116.10:4081 , piece_id=VCIGN7BIDPKG5JPPKYKBI6TPQOCRDHJYGUGU5BD7XS3HMOVQPYDQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: <nil> Oct 7 03:08:55.897933416 (325h37m41.360025909s) [noise=true , address=145.79.227.15:4873 , piece_id=SPQBSAW2X5OPFMRHGU4UTJQI5TY2KGGKOMK3VAWL4TZDCS7HZA7A, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:54168->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:54168->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54168->145.79.227.15:4873: use of closed network connection Oct 7 03:08:55.898040309 (325h37m41.360132782s) [noise=true , address=145.79.227.11:4731 , piece_id=NDM2HDHTCRYPFAG3WMSNXY5MI35AAEPCCKAZE2MY4Y3X2TG24APQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: write tcp 188.166.225.170:59312->145.79.227.11:4731: use of closed network connection; write tcp 188.166.225.170:59312->145.79.227.11:4731: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59312->145.79.227.11:4731: use of closed network connection Oct 7 03:08:55.898117546 (325h37m41.360210019s) [noise=true , address=145.79.227.6:4258 , piece_id=PVDGPHPSYORNDP4DJCMWJLXDM6ZLIO6VPKRY22ONYYOUZCJ5MKMA, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:33738->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:33738->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33738->145.79.227.6:4258: use of closed network connection Oct 7 03:08:55.898139198 (325h37m41.360231671s) [noise=true , address=145.79.227.13:4804 , piece_id=NW5CQES6B7MHKX5TJ37RD66HJTOAUNI6BPJPLHZEZW3BNNU47TJA, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: write tcp 188.166.225.170:56868->145.79.227.13:4804: use of closed network connection; write tcp 188.166.225.170:56868->145.79.227.13:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56868->145.79.227.13:4804: use of closed network connection Oct 7 03:08:55.898180807 (325h37m41.36027329s) [noise=true , address=145.79.227.18:4005 , piece_id=RJ7GL3RN6VKTBMTPVD7MHHCS56LPUMVA2RUYY2DYWXGNNCKXMDPA, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:44170->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:44170->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44170->145.79.227.18:4005: use of closed network connection Oct 7 03:08:55.898276499 (325h37m41.360368982s) [noise=true , address=145.79.227.16:4911 , piece_id=YGGGPFLOBKEQ5UCDWCCMFPBXBHQZ3PWN2GJALMAC7KBKGIUZILTQ, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 188.166.225.170:57752->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:08:55.898390556 (325h37m41.360483029s) [noise=true , address=145.79.227.19:4078 , piece_id=RZ3K6YW2JCWAZKLRAYUTOAH2NKLJIZVFPRNF2M5GZ2QNRODS3P2Q, seg_pos={0 0}, upload=286544]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: read tcp 188.166.225.170:44202->145.79.227.19:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 03:08:55.898416125 (325h37m41.360508608s) [seg_pos={0 0}, upload=286544]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 03:08:56.150921774 (325h37m41.613014257s) [upload=286544]: Done uploading segments... Oct 7 03:08:56.150955829 (325h37m41.613048312s) [upload=286544]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.254642732s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.506989755s FAIL (2025-10-07 02:59:03 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.507025142s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.032440658s FAIL (2025-10-07 02:58:11 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.032474282s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.25728132s FAIL (2025-10-07 02:57:45 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.257307328s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.014991622s FAIL (2025-10-07 02:57:39 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 6.015026529s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)