Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.857878715s
FAIL (2025-07-18 07:45:20 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.857909293s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.72559805s
FAIL (2025-07-18 07:44:15 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.725628127s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.925566063s
FAIL (2025-07-18 07:43:34 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.925600528s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.782465842s
FAIL (2025-07-18 07:43:06 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.782493583s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.781882071s
FAIL (2025-07-18 07:42:05 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.781946553s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.951031136s
FAIL (2025-07-18 07:41:50 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.951063628s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.91151537s
FAIL (2025-07-18 07:40:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.911543274s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.970431361s
FAIL (2025-07-18 07:39:38 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.97045757s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.723161718s
FAIL (2025-07-18 07:38:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.723216893s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.852702941s
FAIL (2025-07-18 07:38:01 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.852727608s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.164603041s
FAIL (2025-07-18 07:37:04 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 6.164632538s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.717191285s
FAIL (2025-07-18 07:36:48 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.717237854s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.740227468s
FAIL (2025-07-18 07:36:22 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.74030229s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.790368819s
FAIL (2025-07-18 07:35:41 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.790396373s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.809478235s
FAIL (2025-07-18 07:35:03 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.809504204s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.695398489s
FAIL (2025-07-18 07:34:21 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.695423277s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.958132907s
FAIL (2025-07-18 07:34:01 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.958157415s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.814151678s
FAIL (2025-07-18 07:33:38 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.814227464s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.8455887s
FAIL (2025-07-18 07:33:14 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.845613426s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.776089491s
FAIL (2025-07-18 07:32:30 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.776117114s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.916477936s
FAIL (2025-07-18 07:32:11 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.916503765s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.761741647s
FAIL (2025-07-18 07:31:47 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.761769242s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.90769575s
FAIL (2025-07-18 07:30:38 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.907722481s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.919500642s
FAIL (2025-07-18 07:30:23 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.919525619s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.890665335s
FAIL (2025-07-18 07:29:16 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.890693148s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.737839728s
FAIL (2025-07-18 07:28:18 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.737867452s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.754830097s
FAIL (2025-07-18 07:27:17 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.754857468s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.842551071s
FAIL (2025-07-18 07:27:01 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.842579546s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.887957055s
FAIL (2025-07-18 07:26:30 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.887982334s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
5.788460957s
FAIL (2025-07-18 07:25:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.78848828s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.898991233s
FAIL (2025-07-18 07:24:42 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.899013896s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.819985301s
FAIL (2025-07-18 07:24:15 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.820012283s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.734479414s
FAIL (2025-07-18 07:23:43 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.734519601s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.779151975s
FAIL (2025-07-18 07:22:32 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.779176331s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.702146499s
FAIL (2025-07-18 07:21:47 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.702169974s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.840347131s
FAIL (2025-07-18 07:21:11 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.840376938s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.801270848s
FAIL (2025-07-18 07:20:37 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.801297019s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.88138282s
FAIL (2025-07-18 07:19:25 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.881407067s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.859698629s
FAIL (2025-07-18 07:18:32 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.859729968s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.040595251s
FAIL (2025-07-18 07:17:46 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 6.040629155s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.76488663s
FAIL (2025-07-18 07:16:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.764909353s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
8.845760249s
FAIL (2025-07-18 07:15:44 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 8.845785398s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.869685339s
FAIL (2025-07-18 07:14:40 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.869712591s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.781209621s
FAIL (2025-07-18 07:13:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.781236833s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.751460193s
FAIL (2025-07-18 07:13:04 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.7514843s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.807581695s
FAIL (2025-07-18 07:12:36 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.807608076s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.755533759s
FAIL (2025-07-18 07:11:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.755564276s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.788718791s
FAIL (2025-07-18 07:11:12 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.788747156s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.773242187s
FAIL (2025-07-18 07:10:17 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.773282205s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.847229567s
FAIL (2025-07-18 07:10:01 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:07:45.696547618 (597h57m20.467188963s) [upload=478223]: Done uploading segments...
Jul 18 07:07:45.696571394 (597h57m20.467212729s) [upload=478223]: Upload finished. err: <nil>
Jul 18 07:08:39.443637302 (597h58m14.214278637s) [upload=478234]: Starting upload
Jul 18 07:08:39.443747633 (597h58m14.214388968s) [upload=478234]: Done starting upload
Jul 18 07:08:39.443822536 (597h58m14.214463872s) [upload=478234]: Uploading segments...
Jul 18 07:08:39.444096972 (597h58m14.214738307s) [upload=478234]: Got next segment. Inline: false
Jul 18 07:08:39.444163860 (597h58m14.214805195s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jul 18 07:08:39.444171154 (597h58m14.214812489s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jul 18 07:08:39.680956658 (597h58m14.451597993s) [seg_pos={0 0}, upload=478234]: Begin upload segment...
Jul 18 07:08:39.680998998 (597h58m14.451640333s) [seg_pos={0 0}, upload=478234]: Done begin upload segment.
Jul 18 07:08:39.681021482 (597h58m14.451662817s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681101324 (597h58m14.451742649s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681376451 (597h58m14.452017786s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681577946 (597h58m14.452219281s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681648121 (597h58m14.452289456s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681915833 (597h58m14.452557168s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.681995987 (597h58m14.452637322s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682170600 (597h58m14.452811935s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682235374 (597h58m14.452876709s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682411231 (597h58m14.453052566s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682579252 (597h58m14.453220597s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682688461 (597h58m14.453329796s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.682956113 (597h58m14.453597448s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683091402 (597h58m14.453732737s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683162598 (597h58m14.453803933s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:39.683342693 (597h58m14.453984028s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Uploading piece...
Jul 18 07:08:41.398412262 (597h58m16.169053607s) [upload=478234]: Next returned nil segment
Jul 18 07:08:41.398446617 (597h58m16.169087953s) [upload=478234]: Waiting for error group managing segments...
Jul 18 07:08:41.731911934 (597h58m16.502553269s) [noise=true , address=104.36.82.7:4911 , piece_id=AX5P3CHQWONGNXELH72P7RIRRDZZHCCLBOXJFMRBPRIYM5LP5UPQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.734454449 (597h58m16.505095784s) [noise=true , address=104.36.82.5:4838 , piece_id=6DEDEZTQQO2WPLVYTRCEEW3Y3P5VFRGZ4TQPV4EKTGPW4E75PIDQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741211768 (597h58m16.511853103s) [noise=true , address=45.91.116.4:4940 , piece_id=QBIPM6PRGSXNJG3SQALKDUUXS7GE6SICF6MBIUANQCNQ4YMWQLGQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.741827566 (597h58m16.512468901s) [noise=true , address=45.91.116.6:4975 , piece_id=PABDRF6ZPFYHREBZPNZFNY5DOXEKKIVGG27KAZS6RDPSFTX7PXQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.744254330 (597h58m16.514895665s) [noise=true , address=45.91.116.7:4009 , piece_id=NIBEWFNWDKA6Z3GONZFTMT7MHE3HMKCOVSLKVSN24I2ADTUC7M2A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.746151199 (597h58m16.516792534s) [noise=true , address=104.36.82.4:4801 , piece_id=GIRCNZSHFPVMQDAI5YTBB353RNTV5OQSKY7VRUYHCNXBYCE3664Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.748519771 (597h58m16.519161106s) [noise=true , address=45.91.116.10:4082 , piece_id=PFGV3A36G6S2LCRECFZVYDSIOIQUC25JQFHZZPWQE7VIJEG36TGA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.755497742 (597h58m16.526139077s) [noise=true , address=45.91.116.9:4262 , piece_id=TQTV2JMVDUY374QO6HTATRZ6VJLCOTXYXQK3Z4UDHNRRLLG76D7A, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.756507835 (597h58m16.52714918s) [noise=true , address=104.36.82.8:4970 , piece_id=ZBVB766XFEIGN5PV5WMVJIBNA2L6MIPNSMIJG7W4YNIAID3CTBQQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.817254835 (597h58m16.58789617s) [noise=true , address=45.91.116.6:4974 , piece_id=VLEECY4YJ7TS7CR5LXFNX3QW7BWTTI4DT7DVCFHJDOB4NHSXEI7Q, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.868484625 (597h58m16.639125961s) [noise=true , address=104.36.82.14:4223 , piece_id=PNYQU3ASFRGEHYKJ7PJXV56HXJD2XNTGSBYPWP7VRKUSII44OACA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:08:41.881325305 (597h58m16.65196664s) [noise=true , address=104.36.82.9:4006 , piece_id=XPGCLYMI3YBW3KYZV5W6RWZTAYNNQRQ7I4D2CTNF47OZLQPA24MQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865470201 (597h58m42.636111536s) [noise=true , address=77.90.31.15:4698 , piece_id=Z35HJLVCIJS6UOSAWZQGEFVGVVBMDYBW3EUZYSIXPRERJN5GHUTQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: <nil>
Jul 18 07:09:07.865495670 (597h58m42.636137005s) [seg_pos={0 0}, upload=478234]: Segment reached optimal threshold of 13 pieces.
Jul 18 07:09:07.865713026 (597h58m42.636354361s) [noise=true , address=77.90.31.19:4843 , piece_id=VINENEVDWMDECREXJ4DRJE6CXKVL3WFXAWNPJWLM7F2R37UGJSWA, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb): protocol: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52032->77.90.31.19:4843: use of closed network connection
Jul 18 07:09:07.865751409 (597h58m42.636392744s) [noise=true , address=77.90.31.11:4554 , piece_id=EV5BHW6IEQCYHQIHMRXWUBIWF5HHLD2RYH7WJVJZVJ7GOZ5SMFXQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59464->77.90.31.11:4554: use of closed network connection
Jul 18 07:09:07.865789863 (597h58m42.636431198s) [noise=true , address=77.90.31.10:4520 , piece_id=I2KF36ZTAALIPY5R67QENP4ZACNGR3DFVHHBNK7YJKYWPE54LMMQ, seg_pos={0 0}, upload=478234]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33062->77.90.31.10:4520: use of closed network connection
Jul 18 07:09:07.865840781 (597h58m42.636482116s) [seg_pos={0 0}, upload=478234]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jul 18 07:09:07.865901747 (597h58m42.636543082s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jul 18 07:09:07.865905675 (597h58m42.63654701s) [upload=478234]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jul 18 07:09:08.180381147 (597h58m42.951022492s) [upload=478234]: Done uploading segments...
Jul 18 07:09:08.180402578 (597h58m42.951043913s) [upload=478234]: Upload finished. err: <nil>
Duration: 5.847262809s
core
uplink
us-select-1
|