Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X.............X.................] duration: 6.111890955s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:32:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.111919461s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X.............X................] duration: 5.782251533s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:31:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.78228136s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X.............X...............] duration: 6.088632221s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:31:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.088648061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.............X..............] duration: 6.104256482s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:30:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.104288796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X...............................X.............X.............] duration: 5.817102299s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:29:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.817132538s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X...............................X.............X............] duration: 5.722613867s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:28:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.722646159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X...............................X.............X...........] duration: 5.918096874s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:27:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.918129004s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X...............................X.............X..........] duration: 5.988391356s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:26:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.988414339s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X...............................X.............X.........] duration: 6.130639774s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:26:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.130675511s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X...............................X.............X........] duration: 6.122664426s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:25:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.122701186s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X...............................X.............X.......] duration: 6.194424s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:25:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.194464978s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X...............................X.............X......] duration: 5.934758848s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:23:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.934786804s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X...............................X.............X.....] duration: 6.024369516s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:22:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.024399874s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X...............................X.............X....] duration: 6.001341223s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:21:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.001371643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X...............................X.............X...] duration: 6.083682355s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:20:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.08371575s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X...............................X.............X..] duration: 6.099382572s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:19:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.099415866s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X...............................X.............X.] duration: 19.255921829s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:19:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 19.255951576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X...............................X.............X] duration: 30.251169864s trace: https://jaeger.dev.storj.io/trace/13318194f07c49b1 FAIL (2025-11-14 01:18:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322476654 (113h14m25.873947105s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.322792486 (113h14m25.874262937s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.323252613 (113h14m25.874723074s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.325651880 (113h14m25.877122341s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.327497866 (113h14m25.878968327s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.329256595 (113h14m25.880727046s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330184582 (113h14m25.881655073s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.330458885 (113h14m25.881929336s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.331590479 (113h14m25.88306093s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.332248402 (113h14m25.883718853s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.333939182 (113h14m25.885409633s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:27.334538603 (113h14m25.886009144s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Uploading piece... Nov 14 01:18:29.065218593 (113h14m27.616689054s) [upload=99655]: Next returned nil segment Nov 14 01:18:29.065236327 (113h14m27.616706788s) [upload=99655]: Waiting for error group managing segments... Nov 14 01:18:29.399196085 (113h14m27.950666546s) [noise=true , address=45.91.116.13:4154 , piece_id=TJDD4FG3JVFHPSSQQHXQC5N5EJVUQ7XGNRZNKI4GJAAVU4CKYUAQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.415674112 (113h14m27.967144563s) [noise=true , address=45.91.116.10:4082 , piece_id=3XWEHYHWOQG7CN32QRZ57TPHK4SWYUX4GYTRCV5Z3D66MCDXPONA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.454275637 (113h14m28.005746098s) [noise=true , address=77.90.31.13:4627 , piece_id=43IKKMJRYSGZUYFDID52ON5YBJ2P42K2HV6JXECVHQ4OKMVPCLQQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.485372290 (113h14m28.036842751s) [noise=true , address=77.90.31.18:4807 , piece_id=YQQ3KG6Y2VXQPBASJOX72BMQ2IRNLJMOZ52GZYZBNDDHQT2WZQLQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.507922209 (113h14m28.05939267s) [noise=true , address=77.90.31.17:4772 , piece_id=DJ5GHNPULJYPEXVCFISWKU752MTYY35LSDM5EKEJUJOID6TS36NA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:29.515169459 (113h14m28.06663994s) [noise=true , address=45.91.116.4:4939 , piece_id=QRIMIGZ4WBCF4OGSZ63EP6N337ZI5I6OVZR6RDZRVSKXTUTZTCRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: <nil> Nov 14 01:18:56.346428790 (113h14m54.897899241s) [noise=true , address=145.79.227.23:4545 , piece_id=AV7AUN3LSZ26T26UNM7BO5ENQITI3RZHOQBLQN5JXV7EPI4BWOPQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra, address:145.79.227.23:4545): protocol: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39754->145.79.227.23:4545: use of closed network connection Nov 14 01:18:56.346570660 (113h14m54.898041121s) [noise=true , address=145.79.227.10:4402 , piece_id=64FTGA6X4GLBFGGYLBBMPLTY7SPZ24PXV2MPAEOBTIQXCY5T3M2A, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59506->145.79.227.10:4402: use of closed network connection Nov 14 01:18:56.346671302 (113h14m54.898141763s) [noise=true , address=145.79.227.30:4761 , piece_id=OEORDJW5EIRC2C36AX5ARMFSL47ZJJLGGVJK3J3MGBXQQ6UM5DQA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48540->145.79.227.30:4761: use of closed network connection Nov 14 01:18:56.347092664 (113h14m54.898563115s) [noise=true , address=145.79.227.8:4329 , piece_id=5M7LUVH6CLCX3US3QYOEYXRQJPYEJ46PCFQ65245BYVM76WULSLA, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: rpc: dial tcp 145.79.227.8:4329: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.347215788 (113h14m54.898686239s) [noise=true , address=145.79.227.16:4911 , piece_id=UBA2UVVELEPSCNHIWFDULZUI7NVGNAVJY75OEQEL7AMTD57UXARQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59562->145.79.227.16:4911: use of closed network connection Nov 14 01:18:56.347484681 (113h14m54.898955132s) [noise=true , address=145.79.227.14:4839 , piece_id=ESMLF6ORZ36GUC7O55UDBOMVFQHH7XU5STGRRITAP233ODAREOXQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:145.79.227.14:4839): protocol: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47848->145.79.227.14:4839: use of closed network connection Nov 14 01:18:56.347682818 (113h14m54.899153279s) [noise=true , address=145.79.227.15:4876 , piece_id=JPYIAXX253BJTTMY2BU5YUMAGQPAJURROUKSRC3WODJZ6CUEQBKQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:34888->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348105974 (113h14m54.899576435s) [noise=true , address=145.79.227.6:4258 , piece_id=GYVSBMD4PWWROXCXGDSRD4LGI75G3UMXTZPER2L4BBQV6PBXL5AQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:46906->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348266058 (113h14m54.899736529s) [noise=true , address=145.79.227.19:4079 , piece_id=UK6W7IVKPVLKNI5N2WH64YRMTQ47JIIBP4ARCGSBSRGGASE54XRQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:34294->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348402949 (113h14m54.89987341s) [noise=true , address=145.79.227.17:4972 , piece_id=6HEJQSSNII4W36WTALFIQWTLGAG67QS6CHNNARD343IDCVFLYCOQ, seg_pos={0 0}, upload=99655]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49098->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:18:56.348443776 (113h14m54.899914227s) [seg_pos={0 0}, upload=99655]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Nov 14 01:18:56.596575523 (113h14m55.148045984s) [upload=99655]: Done uploading segments... Nov 14 01:18:56.596611662 (113h14m55.148082123s) [upload=99655]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.251231071s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...............................X.............] duration: 6.004876526s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:17:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.004901012s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..............X...............................X............] duration: 6.006546976s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:17:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.006577345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X..............X...............................X...........] duration: 5.911839268s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:17:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.911873231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X..............X...............................X..........] duration: 5.802629733s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:16:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.802659859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..............X...............................X.........] duration: 5.910491992s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:16:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.910675731s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X..............X...............................X........] duration: 5.959287516s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:15:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.959409689s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X..............X...............................X.......] duration: 6.000431974s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:14:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.000459235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X..............X...............................X......] duration: 7.04689168s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:14:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 7.046945253s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X..............X...............................X.....] duration: 6.063682102s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:13:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.063709714s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X..............X...............................X....] duration: 6.086958595s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:13:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.08697718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X..............X...............................X...] duration: 6.000061337s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:12:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.000096774s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X..............X...............................X..] duration: 5.907125751s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:11:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 5.90717264s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X..............X...............................X.] duration: 6.146168992s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:10:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 6.146201794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X..............X...............................X] duration: 30.247241051s trace: https://jaeger.dev.storj.io/trace/6539e3da3e5f224f FAIL (2025-11-14 01:09:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== h5m11.543472454s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992482648 (113h5m11.543953149s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992547752 (113h5m11.544018213s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.992954787 (113h5m11.544425238s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993417258 (113h5m11.544887719s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.993872795 (113h5m11.545343256s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994270303 (113h5m11.545740764s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.994661798 (113h5m11.546132259s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995302147 (113h5m11.546772618s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.995698943 (113h5m11.547169404s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996142256 (113h5m11.547612717s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.996550324 (113h5m11.548020785s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997532524 (113h5m11.549002985s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:12.997839589 (113h5m11.54931005s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Uploading piece... Nov 14 01:09:14.767046407 (113h5m13.318516868s) [upload=99514]: Next returned nil segment Nov 14 01:09:14.767064341 (113h5m13.318534802s) [upload=99514]: Waiting for error group managing segments... Nov 14 01:09:15.113640114 (113h5m13.665110575s) [noise=true , address=45.91.116.5:4298 , piece_id=WWGZK2EAPFR4OUP65GVD3AYXS7AMZ7OKEKLOSQTVRRMDGZSBFEFQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.116478789 (113h5m13.66794925s) [noise=true , address=77.90.31.15:4699 , piece_id=IGZOUSYTJ3ZT5FHOZSY72ZZHRIU5645D62WSQKJRFSV7E3S4ACRQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.121175192 (113h5m13.672645653s) [noise=true , address=45.91.116.7:4011 , piece_id=L7W7QK6LXKJZZLMCFNQKGRTTP2YTXB6HHOLRM2FIMDIFUSPDTBWA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.140877799 (113h5m13.69234825s) [noise=true , address=45.91.116.14:4190 , piece_id=CRKZSTBTKX6MKM6CAVIJ2Y3RDTVWMYTYQURTSZRC5UWICLMFYLDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.163361432 (113h5m13.714831893s) [noise=true , address=77.90.31.10:4517 , piece_id=ZNTED6E7Z7R5QLI7Q7PV57CLSNLPQAWP6A5QOC3SUN24KK3KY2VQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.179614681 (113h5m13.731085132s) [noise=true , address=77.90.31.21:4877 , piece_id=TP3IR723L4CXD6LUYCUMP2KMBBM4VDAO7UJ7NLDZT6YRVT4TRQTQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:15.204169949 (113h5m13.75564041s) [noise=true , address=77.90.31.16:4735 , piece_id=G2PP22AATJJQ3VAAXGAIVVFUMPICWF6OP2L7TF4WGJ245B4YQIFA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: <nil> Nov 14 01:09:42.030179921 (113h5m40.581650372s) [noise=true , address=145.79.227.32:4834 , piece_id=UZEPSJRPDRLTCBDRQVZFM2RSCFBLGGXDIM4PJCOMKKXT2K2OIZPQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: read tcp 188.166.225.170:60016->145.79.227.32:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.030231639 (113h5m40.58170211s) [noise=true , address=145.79.227.8:4329 , piece_id=PONG5GYVFADG2BNCDGHBC5NKJVJ7JTOTFPR5PJCG6EP6HCRDVMQA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43420->145.79.227.8:4329: use of closed network connection Nov 14 01:09:42.030428965 (113h5m40.581899416s) [noise=true , address=145.79.227.21:4474 , piece_id=3CSLSPRDUR24WKJPQDLIK7TSTK7XS6B6EEYFL52U7KGTR23FLF7A, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1rL2DnjSgHP36Md3bCgy5padqTAhv3iQCeURbLMuuuMBxtUNZH, address:145.79.227.21:4474): protocol: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->145.79.227.21:4474: use of closed network connection Nov 14 01:09:42.030581776 (113h5m40.582052227s) [noise=true , address=145.79.227.30:4761 , piece_id=JBO3HNRJCA5ZUBIH6GNNZWJ5STELBB6SBYDWI2IGXT5D56OPZVMA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:145.79.227.30:4761): protocol: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51670->145.79.227.30:4761: use of closed network connection Nov 14 01:09:42.030704690 (113h5m40.582175151s) [noise=true , address=145.79.227.28:4727 , piece_id=E7ZXDA2PSDWPW2C36JOAV5HEL76QTQVVWLX3JWP6VG77TLPT6MVA, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52414->145.79.227.28:4727: use of closed network connection Nov 14 01:09:42.030867230 (113h5m40.582337691s) [noise=true , address=145.79.227.11:4730 , piece_id=37RH2HFZFWRBE45S6FA4LBTFCHUWLH7RMGHD35TEHVS3225FCY6Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55838->145.79.227.11:4730: use of closed network connection Nov 14 01:09:42.030919980 (113h5m40.582390441s) [noise=true , address=145.79.227.15:4876 , piece_id=WQ3VEDFZ7MAMGSCQB2ZGE3KI35AYZWSGCLIEPGI37QWPVN5TFVDQ, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:60822->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031017636 (113h5m40.582488087s) [noise=true , address=145.79.227.9:4367 , piece_id=QVUM7U5UQKRFCM7OL3LH2ZDUCSCSE2BZ3E7HZUKQ4K62H2LWD23Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:12CMjNpf3aMdqEYea6xSrAWGoEMzfy4SbEjXywZN7JPRCbvDpK6, address:145.79.227.9:4367): protocol: read tcp 188.166.225.170:38584->145.79.227.9:4367: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031124600 (113h5m40.582595061s) [noise=true , address=145.79.227.14:4838 , piece_id=UACGUZQ7EKUSRV64JTASAYJWEFMJZ44BCFBBU2Q5VF2PBB2IV73Q, seg_pos={0 0}, upload=99514]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:42378->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 01:09:42.031150810 (113h5m40.582621261s) [seg_pos={0 0}, upload=99514]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Nov 14 01:09:42.275780278 (113h5m40.827250739s) [upload=99514]: Done uploading segments... Nov 14 01:09:42.275818621 (113h5m40.827289082s) [upload=99514]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.247393984s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X............X..............X...............................] duration: 6.898840204s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:08:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.898862146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X............X..............X..............................] duration: 5.860893839s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:07:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.86093651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X............X..............X.............................] duration: 6.043856947s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:06:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.043888626s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X............X..............X............................] duration: 6.296435507s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:05:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.296467168s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X............X..............X...........................] duration: 5.859916377s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:05:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.859991401s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X............X..............X..........................] duration: 6.00766704s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:04:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.007702336s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X............X..............X.........................] duration: 5.982148007s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:03:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.982172975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X............X..............X........................] duration: 5.932838653s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:02:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.932867678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X............X..............X.......................] duration: 5.933316083s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:02:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.933354446s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X............X..............X......................] duration: 6.328690165s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:01:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.328719873s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X............X..............X.....................] duration: 5.941416738s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 01:00:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.941450333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X............X..............X....................] duration: 5.95009714s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:59:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.950127318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X............X..............X...................] duration: 5.876358145s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:58:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.876385065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X............X..............X..................] duration: 5.817011209s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:58:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.817046407s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X............X..............X.................] duration: 6.010774268s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:57:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.010804425s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X............X..............X................] duration: 5.921010318s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:56:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.921050734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X............X..............X...............] duration: 5.902294912s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:55:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.902321472s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X............X..............X..............] duration: 6.070182158s trace: https://jaeger.dev.storj.io/trace/72049cb4ff5a09b2 FAIL (2025-11-14 00:55:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 40m57.615227656s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064199728 (112h40m57.615670189s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.064660315 (112h40m57.616130776s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065122846 (112h40m57.616593297s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.065609973 (112h40m57.617080434s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.067195332 (112h40m57.618665793s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.069061495 (112h40m57.620531966s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.070956022 (112h40m57.622426503s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.072963465 (112h40m57.624433926s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075421435 (112h40m57.626891896s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075473995 (112h40m57.626944456s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.075946565 (112h40m57.627417026s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.077247141 (112h40m57.628717602s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:44:59.082390435 (112h40m57.633860896s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Uploading piece... Nov 14 00:45:00.772532456 (112h40m59.324002907s) [upload=99153]: Next returned nil segment Nov 14 00:45:00.772549278 (112h40m59.324019739s) [upload=99153]: Waiting for error group managing segments... Nov 14 00:45:01.174935826 (112h40m59.726406287s) [noise=true , address=77.90.31.15:4699 , piece_id=M4ZU767KDVDG3VU3NJWIWHFA47KHF7VTYVM65HVS3D4VIUXSFRNQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.188389462 (112h40m59.739859924s) [noise=true , address=45.91.116.9:4261 , piece_id=AVKFOVPMRVBFLV67F64IYI6EXQQSTFZDWES6LOXDYS5F5BYFZHUQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.189404615 (112h40m59.740875086s) [noise=true , address=77.90.31.10:4520 , piece_id=QY3IJRNTDQZVDEKPBKYUU65B6S4JEFYEDRVACNRZB7PR6SDZ4I5Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.192727823 (112h40m59.744198294s) [noise=true , address=45.91.116.6:4973 , piece_id=3TIP4L7PDRALIEKAPSDEY4RWGV3JNRJSEEVEFXI3HBOIUWNPN5KA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:01.206850573 (112h40m59.758321054s) [noise=true , address=45.91.116.11:4119 , piece_id=2CW5WG7VDKFGUQIOGWGWDM3OD7NGYOKCRARFMKREEQN42SJ6V2XA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: <nil> Nov 14 00:45:28.048026133 (112h41m26.599496594s) [noise=true , address=145.79.227.21:4475 , piece_id=4BOLMN33BXY5A67JBNEQELV666DYP6H4VWMIVIXUJL3XTNK3DLJA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:128SC59hAFBUVutPFym3iZv9qNYtgiqrk6q9RpfWL2ftyTt5Tzk, address:145.79.227.21:4475): protocol: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32808->145.79.227.21:4475: use of closed network connection Nov 14 00:45:28.048562434 (112h41m26.600032905s) [noise=true , address=145.79.227.20:4439 , piece_id=4RW3V4SI6NOEBT7WO63YUIE67XAVEE37KAPZGOWM6AQM6M3OF2ZA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1wgkbc3Mkej7supxhYMtUdBvn9t45i6kmAokAUkqFxEpehMjAn, address:145.79.227.20:4439): protocol: read tcp 188.166.225.170:48818->145.79.227.20:4439: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048827238 (112h41m26.600297699s) [noise=true , address=145.79.227.9:4366 , piece_id=3JZJC5PBWY4QP5H2LWSAJ6CVM52HJC6QCJWPCWR3U6XINXIRILCA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: read tcp 188.166.225.170:56892->145.79.227.9:4366: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.048985709 (112h41m26.60045617s) [noise=true , address=145.79.227.14:4840 , piece_id=A2T4WPYLMWOTPLVW7R5C2NKKAMGTY7WTHXXKPNTRJDPR6ICHOTWQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:59450->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049116950 (112h41m26.600587411s) [noise=true , address=145.79.227.8:4329 , piece_id=PDC7UQVPT6FROP2EQPIBQS6HBVASNTKF26RQYTTQCU7JVQDTDR4Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:124ht9BFK3Y12gKG8i1Szefj4jkP71hgXZ5XLT4schym7GX5ycW, address:145.79.227.8:4329): protocol: read tcp 188.166.225.170:50352->145.79.227.8:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049154250 (112h41m26.600624712s) [noise=true , address=145.79.227.19:4079 , piece_id=AMR3SVTHY6S73DFLGIATUAZEVWKWGA7TT6NAUGZIN2X4RGGKL4CA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: read tcp 188.166.225.170:52362->145.79.227.19:4079: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049497154 (112h41m26.600967605s) [noise=true , address=145.79.227.17:4970 , piece_id=DHHP2EKISOKA7Y4Z677EMPLKPX3QJRQOZGWSTZ5F4B45C3MNNLFQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:45328->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.049966206 (112h41m26.601436667s) [noise=true , address=145.79.227.8:4331 , piece_id=XEKV5L3FHUPQMSUSXKV7YOPRWU6DKVXIZ5MUDT6Y6OATQHRUBVFA, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1JCBWFkdqHkXb7CR2vdPeHWPuMRRoZT2HWqTa1zDwQ9gwfu8uU, address:145.79.227.8:4331): protocol: read tcp 188.166.225.170:36288->145.79.227.8:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050047301 (112h41m26.601517762s) [noise=true , address=145.79.227.24:4583 , piece_id=6FXRXBQN4RSUC56QANDVW5N5VI2IMGHW5RIBTJ3THAFWLPGGXFYQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:57168->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050252472 (112h41m26.601722933s) [noise=true , address=145.79.227.16:4910 , piece_id=ZUZL2MREUWHXF22O6F452KUB6VDO7FMF7XVZBV4AL2NXELA4CSXQ, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: read tcp 188.166.225.170:52444->145.79.227.16:4910: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050416684 (112h41m26.601887145s) [noise=true , address=145.79.227.23:4547 , piece_id=O77LUJVEZW5BDFOXDGABGLFAC5HOCJALKLE4XL7T3ZGTYMYVTW2Q, seg_pos={0 0}, upload=99153]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: read tcp 188.166.225.170:47644->145.79.227.23:4547: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 14 00:45:28.050502608 (112h41m26.601973069s) [seg_pos={0 0}, upload=99153]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 14 00:45:28.306816345 (112h41m26.858286806s) [upload=99153]: Done uploading segments... Nov 14 00:45:28.306886328 (112h41m26.858356779s) [upload=99153]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.070216214s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)