Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X....X.X....................................................] duration: 6.010332952s FAIL (2025-10-07 09:47:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.010362419s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 5.810492886s FAIL (2025-10-07 09:47:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.810522664s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XX....X.X...................................................] duration: 6.320629129s FAIL (2025-10-07 09:47:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.320665679s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 5.791569364s FAIL (2025-10-07 09:46:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.791602909s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [XXX....X.X..................................................] duration: 5.965448504s FAIL (2025-10-07 09:45:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.965473923s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 5.777276217s FAIL (2025-10-07 09:45:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.777318647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [XXXX....X.X.................................................] duration: 6.003338505s FAIL (2025-10-07 09:45:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.003372069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XXXXX....X.X................................................] duration: 6.081955375s FAIL (2025-10-07 09:44:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.081990742s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XXXX...................................................] duration: 5.809037034s FAIL (2025-10-07 09:44:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.809074667s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XXXXXX....X.X...............................................] duration: 6.188727641s FAIL (2025-10-07 09:44:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.188760793s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XXXX..................................................] duration: 5.991616967s FAIL (2025-10-07 09:43:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.991650821s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......XXXX.................................................] duration: 5.888909707s FAIL (2025-10-07 09:43:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.888941177s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXXXXXX....X.X..............................................] duration: 5.91889456s FAIL (2025-10-07 09:43:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.918934316s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........XXXX................................................] duration: 6.027914867s FAIL (2025-10-07 09:43:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.027947789s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXXXXXXX....X.X.............................................] duration: 6.177814868s FAIL (2025-10-07 09:42:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.177861577s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........XXXX...............................................] duration: 5.775217224s FAIL (2025-10-07 09:42:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.775244647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........XXXX..............................................] duration: 5.895722171s FAIL (2025-10-07 09:41:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.895757038s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXXXXXXX....X.X............................................] duration: 6.070962764s FAIL (2025-10-07 09:41:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.070998562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.XXXXXXXXX....X.X...........................................] duration: 6.302944952s FAIL (2025-10-07 09:40:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.30297015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XXXX.............................................] duration: 5.83144953s FAIL (2025-10-07 09:40:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.831481952s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XXXXXXXXX....X.X..........................................] duration: 6.034524215s FAIL (2025-10-07 09:40:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.034566657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...XXXXXXXXX....X.X.........................................] duration: 6.113535901s FAIL (2025-10-07 09:39:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.113576979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XXXX............................................] duration: 5.840761263s FAIL (2025-10-07 09:39:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.840798494s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............XXXX...........................................] duration: 5.891589112s FAIL (2025-10-07 09:39:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.891610132s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....XXXXXXXXX....X.X........................................] duration: 6.220661855s FAIL (2025-10-07 09:39:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.220719444s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration: 5.83182809s FAIL (2025-10-07 09:38:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.831862294s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....XXXXXXXXX....X.X.......................................] duration: 6.641997886s FAIL (2025-10-07 09:37:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.642027805s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration: 5.748043874s FAIL (2025-10-07 09:37:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.748082556s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......XXXXXXXXX....X.X......................................] duration: 6.200059239s FAIL (2025-10-07 09:37:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.200126336s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................XXXX........................................] duration: 5.876153632s FAIL (2025-10-07 09:36:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.876185113s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X......XXXXXXXXX....X.X.....................................] duration: 6.440018075s FAIL (2025-10-07 09:36:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.440092788s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................XXXX.......................................] duration: 5.856376346s FAIL (2025-10-07 09:35:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.856478541s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X......XXXXXXXXX....X.X....................................] duration: 6.241183977s FAIL (2025-10-07 09:35:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.241226707s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................XXXX......................................] duration: 5.872286034s FAIL (2025-10-07 09:34:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.872320078s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X......XXXXXXXXX....X.X...................................] duration: 6.330334796s FAIL (2025-10-07 09:34:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.330364594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XXXX.....................................] duration: 5.607549885s FAIL (2025-10-07 09:33:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.607579573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X......XXXXXXXXX....X.X..................................] duration: 6.045501147s FAIL (2025-10-07 09:33:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.045534139s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XXXX....................................] duration: 6.076424562s FAIL (2025-10-07 09:32:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.07644415s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X......XXXXXXXXX....X.X.................................] duration: 5.911942514s FAIL (2025-10-07 09:32:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.911999954s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.....X......XXXXXXXXX....X.X................................] duration: 6.205065232s FAIL (2025-10-07 09:32:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.20512685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XXXX...................................] duration: 6.228536983s FAIL (2025-10-07 09:32:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.228573013s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......X......XXXXXXXXX....X.X...............................] duration: 6.28305658s FAIL (2025-10-07 09:31:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.283092116s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXXX..................................] duration: 5.873755969s FAIL (2025-10-07 09:31:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.873806435s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......X......XXXXXXXXX....X.X..............................] duration: 6.386309743s FAIL (2025-10-07 09:31:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.386346865s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........X......XXXXXXXXX....X.X.............................] duration: 6.252270001s FAIL (2025-10-07 09:30:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.252306s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXXX.................................] duration: 6.317893734s FAIL (2025-10-07 09:30:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.317930694s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........X......XXXXXXXXX....X.X............................] duration: 6.219712293s FAIL (2025-10-07 09:30:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.219753202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXXX................................] duration: 5.988617953s FAIL (2025-10-07 09:29:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.988653582s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........X......XXXXXXXXX....X.X...........................] duration: 6.148980844s FAIL (2025-10-07 09:29:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.149014326s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...........X......XXXXXXXXX....X.X..........................] duration: 6.469516015s FAIL (2025-10-07 09:28:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== 862]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:11:03.928371192 (331h39m49.390463675s) [upload=291862]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:11:04.183677820 (331h39m49.645770333s) [seg_pos={0 0}, upload=291862]: Begin upload segment... Oct 7 09:11:04.183747542 (331h39m49.645840025s) [seg_pos={0 0}, upload=291862]: Done begin upload segment. Oct 7 09:11:04.183784052 (331h39m49.645876535s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186724632 (331h39m49.648817125s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.186780849 (331h39m49.648873342s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187470842 (331h39m49.649563325s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187734805 (331h39m49.649827288s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.187933975 (331h39m49.650026458s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188351951 (331h39m49.650444434s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188575526 (331h39m49.650668009s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.188851722 (331h39m49.650944215s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189354690 (331h39m49.651447173s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.189409173 (331h39m49.651501656s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.190518055 (331h39m49.652610538s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191212127 (331h39m49.65330461s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191722729 (331h39m49.653815212s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.191961554 (331h39m49.654054037s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:04.192666086 (331h39m49.654758569s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Uploading piece... Oct 7 09:11:05.893138363 (331h39m51.355230846s) [upload=291862]: Next returned nil segment Oct 7 09:11:05.893167920 (331h39m51.355260403s) [upload=291862]: Waiting for error group managing segments... Oct 7 09:11:06.280910675 (331h39m51.743003158s) [noise=true , address=77.90.31.21:4877 , piece_id=7PLLMSZHTD54KJ4MT7OKICWGDOLYGHDYRYU57VX6XQVGLSBD7VDA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.295069467 (331h39m51.75716195s) [noise=true , address=45.91.116.8:4045 , piece_id=D2VGVBW6EDT52MMS6A7MKOVCVRDJVTOOEHDJYUK7VBZODMCX744Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.306611055 (331h39m51.768703538s) [noise=true , address=45.91.116.6:4973 , piece_id=6TLSCOMWBYHAACQN7UCPIJRGVHB6OJSLGDIGOEYX5FSHZI6OZPXA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.309047524 (331h39m51.771140007s) [noise=true , address=45.91.116.7:4009 , piece_id=XEJLUB5QR46G532MPICVKRXJECCTG6INXXGT4KNTGCAMQLHK4YHA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.337702319 (331h39m51.799794802s) [noise=true , address=77.90.31.13:4627 , piece_id=OACTFG2OSNQKLNDEEV7HDH4OIYJJHD2DY36JOSU34UOV4R7DCOUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.393571505 (331h39m51.855663988s) [noise=true , address=45.91.116.10:4082 , piece_id=2YNTFBBCDREMTMCHDEPPYNTQWFG4TU57GWT2YE56MC5XNQXDKKYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.420937305 (331h39m51.883029789s) [noise=true , address=77.90.31.17:4770 , piece_id=LEX6UP4SW3FKACTR4MIMPA3I2BUJVLF3LW7X7JCZEDRI7EOLQTWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.455144513 (331h39m51.917236996s) [noise=true , address=45.91.116.14:4190 , piece_id=TNHBTN7YZ4CFCCZYSS6ULVFNU6F7P5XEB5BCH6XS3TO6VKZJFY3Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:06.491218756 (331h39m51.95331124s) [noise=true , address=45.91.116.6:4974 , piece_id=GAOCEHDYO2QBCO47JN34256PEDG4SX73KXBAAUPSEZRY2C6ITT7Q, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:17.404394730 (331h40m2.866487213s) [noise=true , address=145.79.227.16:4909 , piece_id=WWWARDCSM6PVJVA4TG3MISZZJBFBMJCGGMIQRWJSJULTNQCHGBUQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: <nil> Oct 7 09:11:33.217431459 (331h40m18.679523952s) [noise=true , address=145.79.227.19:4078 , piece_id=Z3LFJPGYUOLUBP3VR5OX5BT5TJWYAAJI5WQPUFOMG5DNV7MJZYLA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33480->145.79.227.19:4078: use of closed network connection Oct 7 09:11:33.217451567 (331h40m18.67954405s) [noise=true , address=145.79.227.7:4295 , piece_id=AKSEANJPXQBU6R6OORYAELTQ5VRRAX7EJ7EISSMX5OG7JX3XL7TA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51442->145.79.227.7:4295: use of closed network connection Oct 7 09:11:33.217554052 (331h40m18.679646536s) [noise=true , address=145.79.227.3:4150 , piece_id=C72ZQOAOQGWJTDLM34UJTUF4GWZTSJV6TUFJPBLQ4YZ7FN5FKE5A, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 09:11:33.217581615 (331h40m18.679674098s) [noise=true , address=145.79.227.6:4259 , piece_id=NCMAEQQACTTYJKHPA23KBZN4EBMHIKVCVD3YLMHJOTHB2I3IKOYQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59904->145.79.227.6:4259: use of closed network connection Oct 7 09:11:33.217676685 (331h40m18.679769179s) [noise=true , address=145.79.227.3:4151 , piece_id=FMIRGUFG37VWVTMD7LIJGF6RFBEE6THGDWGHED54I7EPOY6EJAZA, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44116->145.79.227.3:4151: use of closed network connection Oct 7 09:11:33.217768410 (331h40m18.679860894s) [noise=true , address=145.79.227.5:4222 , piece_id=S3ICKBJLSGTLD4ABJ2LXIN3WWT27LQEXCMV2PQ6LWEVGJBOJIZWQ, seg_pos={0 0}, upload=291862]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53934->145.79.227.5:4222: use of closed network connection Oct 7 09:11:33.217857971 (331h40m18.679950455s) [seg_pos={0 0}, upload=291862]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:11:33.464963703 (331h40m18.927056186s) [upload=291862]: Done uploading segments... Oct 7 09:11:33.464995844 (331h40m18.927088317s) [upload=291862]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.469571049s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)