Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.761741647s FAIL (2025-07-18 07:31:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.761769242s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.90769575s FAIL (2025-07-18 07:30:38 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.907722481s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.919500642s FAIL (2025-07-18 07:30:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.919525619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.890665335s FAIL (2025-07-18 07:29:16 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.890693148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.737839728s FAIL (2025-07-18 07:28:18 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.737867452s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.754830097s FAIL (2025-07-18 07:27:17 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.754857468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.842551071s FAIL (2025-07-18 07:27:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.842579546s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.887957055s FAIL (2025-07-18 07:26:30 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.887982334s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.788460957s FAIL (2025-07-18 07:25:49 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.78848828s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.898991233s FAIL (2025-07-18 07:24:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.899013896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.819985301s FAIL (2025-07-18 07:24:15 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.820012283s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.734479414s FAIL (2025-07-18 07:23:43 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.734519601s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.779151975s FAIL (2025-07-18 07:22:32 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.779176331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.702146499s FAIL (2025-07-18 07:21:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.702169974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.840347131s FAIL (2025-07-18 07:21:11 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.840376938s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.801270848s FAIL (2025-07-18 07:20:37 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.801297019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.88138282s FAIL (2025-07-18 07:19:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.881407067s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.859698629s FAIL (2025-07-18 07:18:32 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.859729968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.040595251s FAIL (2025-07-18 07:17:46 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 6.040629155s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.76488663s FAIL (2025-07-18 07:16:53 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.764909353s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 8.845760249s FAIL (2025-07-18 07:15:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 8.845785398s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.869685339s FAIL (2025-07-18 07:14:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.869712591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.781209621s FAIL (2025-07-18 07:13:53 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.781236833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.751460193s FAIL (2025-07-18 07:13:04 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.7514843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.807581695s FAIL (2025-07-18 07:12:36 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.807608076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.755533759s FAIL (2025-07-18 07:11:53 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.755564276s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.788718791s FAIL (2025-07-18 07:11:12 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.788747156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.773242187s FAIL (2025-07-18 07:10:17 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.773282205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.847229567s FAIL (2025-07-18 07:10:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.847262809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.794128666s FAIL (2025-07-18 07:09:32 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 5.794163833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000982811s FAIL (2025-07-18 07:09:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== : pb.BatchRequestItem_ObjectCommit Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments... Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil> Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments... Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment... Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment. Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece... Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments... Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil> Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces. Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments... Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil> Duration:* 30.000999933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.761155916s FAIL (2025-07-18 03:08:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.76118335s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.816857432s FAIL (2025-07-18 03:07:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.816887179s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 5.740950735s FAIL (2025-07-18 03:07:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.740978049s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 5.808801853s FAIL (2025-07-18 03:06:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.808829255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 5.745570733s FAIL (2025-07-18 03:05:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.745598006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 5.814363662s FAIL (2025-07-18 03:04:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.814383591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.816353499s FAIL (2025-07-18 03:04:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.816380349s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.811991347s FAIL (2025-07-18 03:04:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.812025001s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 5.83014518s FAIL (2025-07-18 03:03:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.830168224s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 5.678708567s FAIL (2025-07-18 03:03:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.678733786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [XX.X........................................................] duration: 5.839424168s FAIL (2025-07-18 03:03:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.839470206s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 5.917829706s FAIL (2025-07-18 03:02:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.917853721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.XX.X.......................................................] duration: 5.594544884s FAIL (2025-07-18 03:02:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.594567606s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XXXX...................................................] duration: 5.791903501s FAIL (2025-07-18 03:02:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.791931233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XXXX..................................................] duration: 5.870621206s FAIL (2025-07-18 03:01:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.870644541s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..XX.X......................................................] duration: 5.650288803s FAIL (2025-07-18 03:01:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.650314294s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......XXXX.................................................] duration: 5.988860638s FAIL (2025-07-18 03:01:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.988892639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...XX.X.....................................................] duration: 6.052011278s FAIL (2025-07-18 03:01:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.052039072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....XX.X....................................................] duration: 5.73795106s FAIL (2025-07-18 03:00:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.737990215s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)