Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.415294629s FAIL (2025-06-27 19:37:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.415320289s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.805505037s FAIL (2025-06-27 19:37:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.805529775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.127337493s FAIL (2025-06-27 19:36:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.127364964s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.801349273s FAIL (2025-06-27 19:34:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.801381715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.546910989s FAIL (2025-06-27 19:33:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.546941907s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.186607189s FAIL (2025-06-27 19:32:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.186637486s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.985185112s FAIL (2025-06-27 19:32:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.985215029s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.183673997s FAIL (2025-06-27 19:31:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.18370246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.694991413s FAIL (2025-06-27 19:31:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.695020008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.782810687s FAIL (2025-06-27 19:30:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.782825977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.895813156s FAIL (2025-06-27 19:29:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.895839497s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.080719947s FAIL (2025-06-27 19:28:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.080741708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 6.522624904s FAIL (2025-06-27 19:27:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.522652578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.14152433s FAIL (2025-06-27 19:27:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.141549208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.01253666s FAIL (2025-06-27 19:26:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.012564492s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.806124486s FAIL (2025-06-27 19:25:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.806168471s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.335325336s FAIL (2025-06-27 19:24:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.33535431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.243113473s FAIL (2025-06-27 19:24:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.24316431s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.541964997s FAIL (2025-06-27 19:23:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.541994164s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.063338169s FAIL (2025-06-27 19:22:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.063365732s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.710544106s FAIL (2025-06-27 19:21:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.710571668s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.38969839s FAIL (2025-06-27 19:20:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.389723188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.790786313s FAIL (2025-06-27 19:20:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.790811931s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.896738908s FAIL (2025-06-27 19:19:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.896770108s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.649386553s FAIL (2025-06-27 19:18:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.649412213s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.757286517s FAIL (2025-06-27 19:18:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.757313498s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.746283509s FAIL (2025-06-27 19:18:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.746315131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.602308061s FAIL (2025-06-27 19:17:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.602331035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.982359565s FAIL (2025-06-27 19:16:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.982389383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.843804953s FAIL (2025-06-27 19:16:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.843828047s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.798383889s FAIL (2025-06-27 19:15:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.79841033s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.778717316s FAIL (2025-06-27 19:14:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.778743417s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.680761721s FAIL (2025-06-27 19:13:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.680789374s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.57881059s FAIL (2025-06-27 19:12:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.57883207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.681439707s FAIL (2025-06-27 19:11:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.681465656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.938874681s FAIL (2025-06-27 19:10:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.938897523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.809295193s FAIL (2025-06-27 19:09:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.809322315s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.2677123s FAIL (2025-06-27 19:08:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.267729643s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.701413616s FAIL (2025-06-27 19:07:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.701437833s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.895734587s FAIL (2025-06-27 19:07:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.895760537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.65799486s FAIL (2025-06-27 19:06:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.658023835s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.476659931s FAIL (2025-06-27 19:06:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.476690508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.048229381s FAIL (2025-06-27 19:05:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.048250783s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.692476733s FAIL (2025-06-27 19:05:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.692510096s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.71464074s FAIL (2025-06-27 19:04:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.714664664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.670393381s FAIL (2025-06-27 19:03:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.670420484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.969677236s FAIL (2025-06-27 19:02:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.969698056s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.716927773s FAIL (2025-06-27 19:01:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.716953653s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.865124257s FAIL (2025-06-27 19:01:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.865153053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 8.888851123s FAIL (2025-06-27 19:00:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== equestItem_ObjectCommit Jun 27 18:59:10.869381757 (105h48m45.640023092s) [upload=84602]: Done uploading segments... Jun 27 18:59:10.869401044 (105h48m45.640042379s) [upload=84602]: Upload finished. err: <nil> Jun 27 18:59:46.076627182 (105h49m20.847268517s) [upload=84613]: Starting upload Jun 27 18:59:46.076690603 (105h49m20.847331928s) [upload=84613]: Done starting upload Jun 27 18:59:46.076789222 (105h49m20.847430557s) [upload=84613]: Uploading segments... Jun 27 18:59:46.076864626 (105h49m20.847505961s) [upload=84613]: Got next segment. Inline: false Jun 27 18:59:46.076893341 (105h49m20.847534676s) [upload=84613]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jun 27 18:59:46.076897660 (105h49m20.847538995s) [upload=84613]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jun 27 18:59:46.412760284 (105h49m21.183401619s) [seg_pos={0 0}, upload=84613]: Begin upload segment... Jun 27 18:59:46.412810951 (105h49m21.183452286s) [seg_pos={0 0}, upload=84613]: Done begin upload segment. Jun 27 18:59:46.412835087 (105h49m21.183476412s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.412875254 (105h49m21.183516579s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413251073 (105h49m21.183892408s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413392984 (105h49m21.184034319s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413592907 (105h49m21.184234242s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.413919572 (105h49m21.184560907s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.414803052 (105h49m21.185444387s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415648740 (105h49m21.186290075s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.415977559 (105h49m21.186618894s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416499979 (105h49m21.187141314s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416773242 (105h49m21.187414577s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.416986290 (105h49m21.187627625s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417212041 (105h49m21.187853376s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417425690 (105h49m21.188067015s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417641372 (105h49m21.188282697s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:46.417981784 (105h49m21.188623119s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Uploading piece... Jun 27 18:59:48.179573789 (105h49m22.950215124s) [upload=84613]: Next returned nil segment Jun 27 18:59:48.179591102 (105h49m22.950232437s) [upload=84613]: Waiting for error group managing segments... Jun 27 18:59:48.466479713 (105h49m23.237121038s) [noise=true , address=45.91.116.9:5261 , piece_id=PHQ6KCDGGJQS6AZJG2OGKWGEGHXAGP4UWWFYZV4AHHSY4WEN54NA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.475171252 (105h49m23.245812577s) [noise=true , address=45.91.116.5:5298 , piece_id=F4CG6YJWSDDRPTTENMC7NEKGLHMIPZIOMDQAV5CSHGPUJHMWF6IA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.494180804 (105h49m23.264822139s) [noise=true , address=77.90.31.10:4520 , piece_id=SMQT45XDJPPYXDQUDHNJC23N4CKXER7SH2G65VYXSF4X4AGCH3PA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504253477 (105h49m23.274894812s) [noise=true , address=77.90.31.11:4555 , piece_id=PLMKP4EBU6Q2UT7ANWT2TADTG465HHGQEMV7WKVXPXVHJ2HKF36A, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.504779574 (105h49m23.275420909s) [noise=true , address=77.90.31.10:4519 , piece_id=QLJL6AQYSGW5E5BDBHSTN5CZKZHIXTUFJTZ5PVQVNF6KQTBQJYWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.509122774 (105h49m23.279764109s) [noise=true , address=77.90.31.11:4553 , piece_id=IGS56L57J4VTAWNZCLIGLGVEH5KOXG2BDXQUAUHPOPXYC2FAKW5Q, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.556699455 (105h49m23.32734079s) [noise=true , address=77.90.31.14:4661 , piece_id=WIVURAURMR2IRNYAVP7RDDVR7O7N4BMLGOJQNO2QIBAXXDIR4JWQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.583625665 (105h49m23.354267s) [noise=true , address=77.90.31.18:4806 , piece_id=4DYXB72BFMCGV23HRWO2IAMFQ4IO4UZXWA7KOCTSYJOYO2YPO3UA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.594874819 (105h49m23.365516144s) [noise=true , address=45.91.116.6:4973 , piece_id=WZDBILSUP55YTIBB7I4XTLQ5VNIWDHZYOFS67QKBFFZSAHZXF3RA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.595000279 (105h49m23.365641604s) [noise=true , address=45.91.116.13:4155 , piece_id=T7ICJWQZRQB5HMRPP2PVAXJPIGCQAIPDBV3QGXJ6BRGKA4ZSILGA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 18:59:48.825355520 (105h49m23.595996885s) [noise=true , address=77.90.31.15:4699 , piece_id=HWOHW4FGTAYASBD3MPJQDSF2LH3VDIMJEXKEGOPD4AXWUGPTABIQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:03.146543929 (105h49m37.917185274s) [noise=true , address=104.36.82.6:4875 , piece_id=QTMJOXLYIU7NRSRHS23EH75B65INBE6IYDNOQ4WOMFJSSUHGEGCQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: <nil> Jun 27 19:00:15.353432335 (105h49m50.12407367s) [noise=true , address=104.36.82.14:4223 , piece_id=XPJBXIZOPZMTABM2ZLN7FQCKE4VOBAGVRBNBSM52FUUZFPLYKTJQ, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53868->104.36.82.14:4223: use of closed network connection Jun 27 19:00:15.353463896 (105h49m50.124105231s) [noise=true , address=104.36.82.13:4186 , piece_id=G33JU3ZSBUJCXSOGXENHDFIY2OPGUJLQ3G2UJ7DOOJ27QEANUFUA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46516->104.36.82.13:4186: use of closed network connection Jun 27 19:00:15.353511327 (105h49m50.124152662s) [noise=true , address=104.36.82.4:4804 , piece_id=LWYRHPD6BX4HRW7PMSPN37JFJYF555UHUDJR5X6NEZ6NPN7CH6EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47764->104.36.82.4:4804: use of closed network connection Jun 27 19:00:15.353550963 (105h49m50.124192308s) [noise=true , address=104.36.82.8:4969 , piece_id=EV7URGEFFDSTNTEHVQ2VAXDRWMXILZKBW7BARN34UOUNCL4UM7EA, seg_pos={0 0}, upload=84613]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34188->104.36.82.8:4969: use of closed network connection Jun 27 19:00:15.353588665 (105h49m50.12423001s) [seg_pos={0 0}, upload=84613]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jun 27 19:00:15.585388171 (105h49m50.356029546s) [upload=84613]: Done uploading segments... Jun 27 19:00:15.585474816 (105h49m50.356116151s) [upload=84613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.888870019s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)