Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.614340519s FAIL (2025-07-19 05:18:35 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 6.614363913s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [................XX...X...X....X..X....XXXXX.X.X.....XX.X....] duration: 6.943225083s FAIL (2025-07-19 05:18:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.943256343s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.695963941s FAIL (2025-07-19 05:18:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.695988086s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [.................XX...X...X....X..X....XXXXX.X.X.....XX.X...] duration: 5.861284015s FAIL (2025-07-19 05:17:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.861307069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.611207288s FAIL (2025-07-19 05:16:55 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.611237685s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..................XX...X...X....X..X....XXXXX.X.X.....XX.X..] duration: 6.933027984s FAIL (2025-07-19 05:16:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.933058504s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...................XX...X...X....X..X....XXXXX.X.X.....XX.X.] duration: 19.002634454s FAIL (2025-07-19 05:16:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 19.002654132s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 15.68235027s FAIL (2025-07-19 05:16:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 15.682377171s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.54229691s FAIL (2025-07-19 05:15:51 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.542319122s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [....................XX...X...X....X..X....XXXXX.X.X.....XX.X] duration: 30.23968037s FAIL (2025-07-19 05:15:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== gment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:14:35.936262577 (6h23m42.308841932s) [upload=5075]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:14:35.936281884 (6h23m42.308861239s) [upload=5075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:14:36.234171969 (6h23m42.606751314s) [upload=5075]: Done uploading segments... Jul 19 05:14:36.234192037 (6h23m42.606771382s) [upload=5075]: Upload finished. err: <nil> Jul 19 05:14:55.587986728 (6h24m1.960566093s) [upload=5081]: Starting upload Jul 19 05:14:55.588051823 (6h24m1.960631178s) [upload=5081]: Done starting upload Jul 19 05:14:55.588146724 (6h24m1.960726079s) [upload=5081]: Uploading segments... Jul 19 05:14:55.588391873 (6h24m1.960971228s) [upload=5081]: Got next segment. Inline: false Jul 19 05:14:55.588649837 (6h24m1.961229193s) [upload=5081]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:14:55.588655057 (6h24m1.961234402s) [upload=5081]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:14:55.823387692 (6h24m2.195967047s) [seg_pos={0 0}, upload=5081]: Begin upload segment... Jul 19 05:14:55.823450572 (6h24m2.196029927s) [seg_pos={0 0}, upload=5081]: Done begin upload segment. Jul 19 05:14:55.823481952 (6h24m2.196061307s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823491401 (6h24m2.196070756s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823923789 (6h24m2.196503144s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.823949737 (6h24m2.196529092s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824310158 (6h24m2.196889513s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824403045 (6h24m2.19698239s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824651210 (6h24m2.197230565s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824716334 (6h24m2.197295689s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824960491 (6h24m2.197539856s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.824977494 (6h24m2.197556849s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825213285 (6h24m2.19779264s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825271547 (6h24m2.197850902s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825445860 (6h24m2.198025215s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825568244 (6h24m2.198147599s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.825902985 (6h24m2.19848234s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:55.826216495 (6h24m2.19879585s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Uploading piece... Jul 19 05:14:58.633690556 (6h24m5.006269911s) [upload=5081]: Next returned nil segment Jul 19 05:14:58.633708100 (6h24m5.006287445s) [upload=5081]: Waiting for error group managing segments... Jul 19 05:14:58.915966948 (6h24m5.288546293s) [noise=true , address=45.91.116.7:4009 , piece_id=HH5I32ZYT5LVXHDLBVSAVX6T23MJ6RGNPNAWPIQAREJ4ELPFJGYA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.916093541 (6h24m5.288672896s) [noise=true , address=45.91.116.12:4227 , piece_id=QB7YKZFXWMRB3VQ3HSFMMMLLX5YFWOXG4Y4GJGGYT6HDV4FQ65VA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.923615462 (6h24m5.296194817s) [noise=true , address=45.91.116.14:4190 , piece_id=RUU2MICSAWOECAAYNOBS75SE2YKSUBMSSFKLPON7FWK77UNJEYGQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.928612934 (6h24m5.301192289s) [noise=true , address=45.91.116.7:4010 , piece_id=BH76RM4NXGEAI2J2424ALK4PFSXUTBCYOH777BUOIDW7BW2IJUJA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.990915468 (6h24m5.363494833s) [noise=true , address=45.91.116.4:4937 , piece_id=V2IOOA7DVCGZBH3D43ROYHT6ZECPY6GTC5M7NFO2RWPQB6EYGALA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.992136464 (6h24m5.364715819s) [noise=true , address=45.91.116.8:4047 , piece_id=6UOFOV4VCVT6DB7D7Q23REAX7VZPAW5PSG5PLA54LISOUWMPQBBQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:58.997711812 (6h24m5.370291157s) [noise=true , address=45.91.116.9:4262 , piece_id=FRIATDM7LPPHXYAMUISAYRTQWAKSTB4EYUANHGEMQHM6JQADDTRQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.010548391 (6h24m5.383127746s) [noise=true , address=45.91.116.10:4082 , piece_id=HZDMC4J5O5UHVDH5SOQBKAFPN6RJUHIWKEZ4OPVMW2HSN7NDZMZA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:14:59.039607621 (6h24m5.412186996s) [noise=true , address=45.91.116.5:4299 , piece_id=JQLK5LIMCMW6R2UZEJZSG6QAP2M65XBVE6JFEXY5KDWBVNDG2VCQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:19.998482215 (6h24m26.37106157s) [noise=true , address=77.90.31.14:4663 , piece_id=LNTFUWB6LUUFD2EB536ZHBSNVVXYZPOOB4DUZB2M6BXSSFEL3UMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:21.170089451 (6h24m27.542668806s) [noise=true , address=77.90.31.13:4627 , piece_id=E7GSSBZ653K2AA4TP6A4BRT5BDZCBODNRAZM3P6SYJXJFQU2BACQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: <nil> Jul 19 05:15:24.941469148 (6h24m31.314048503s) [noise=true , address=77.90.31.15:4697 , piece_id=W62ARDAVLI64P6VYSPVFRTDKB5H2WXTSCF2VKP3UN7JKGBCUEZTQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941530766 (6h24m31.314110121s) [noise=true , address=77.90.31.17:4772 , piece_id=B544E3F2UC3UVE2ITUQAWGVDMSPS2HFIVHXZ2OLWB3T25AHMFAVA, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44456->77.90.31.17:4772: use of closed network connection Jul 19 05:15:24.941543591 (6h24m31.314122946s) [noise=true , address=77.90.31.16:4734 , piece_id=YUW6MWSUDNLAC2VHLBT6H4T27YTNJ3RDK4BSGE7OZQCO3F2LBH4A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:15:24.941614786 (6h24m31.314194141s) [noise=true , address=77.90.31.11:4555 , piece_id=YICA5IVHSCC3RPC5FRKD26BSMY6N5WJQA5LCABJSRJYL6BZJYL6A, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37392->77.90.31.11:4555: use of closed network connection Jul 19 05:15:24.941641888 (6h24m31.314221243s) [noise=true , address=77.90.31.18:4806 , piece_id=67PXFJMJB6XWNU6JMOKPO4BXKDD2Z225ZKTB7DZEA46OMSSXHHMQ, seg_pos={0 0}, upload=5081]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47114->77.90.31.18:4806: use of closed network connection Jul 19 05:15:24.941663099 (6h24m31.314242454s) [seg_pos={0 0}, upload=5081]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:15:25.179949476 (6h24m31.552528831s) [upload=5081]: Done uploading segments... Jul 19 05:15:25.179975797 (6h24m31.552555152s) [upload=5081]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.239699457s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 21.867266141s FAIL (2025-07-19 05:15:10 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 21.867295738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....................XX...X...X....X..X....XXXXX.X.X.....XX.] duration: 5.929817875s FAIL (2025-07-19 05:14:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== network connection; write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection Jul 19 05:13:30.499725691 (6h22m36.872305046s) [seg_pos={0 0}, upload=5061]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:13:30.730093449 (6h22m37.102672804s) [upload=5061]: Done uploading segments... Jul 19 05:13:30.730128235 (6h22m37.10270759s) [upload=5061]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:13:31.615702822 (6h22m37.988282217s) [upload=5064]: Starting upload Jul 19 05:13:31.615841878 (6h22m37.988421233s) [upload=5064]: Done starting upload Jul 19 05:13:31.615921711 (6h22m37.988501106s) [upload=5064]: Uploading segments... Jul 19 05:13:31.616305465 (6h22m37.98888485s) [upload=5064]: Got next segment. Inline: false Jul 19 05:13:31.616375579 (6h22m37.988954964s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:13:31.616384025 (6h22m37.988963421s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:13:31.872093568 (6h22m38.244672923s) [seg_pos={0 0}, upload=5064]: Begin upload segment... Jul 19 05:13:31.872131390 (6h22m38.244710735s) [seg_pos={0 0}, upload=5064]: Done begin upload segment. Jul 19 05:13:31.872182527 (6h22m38.244761883s) [noise=true , address=77.90.31.14:4662 , piece_id=7GKZYRTV7JJFMKEE3HHZKGRBBK22UHWS6HF3MGGNPHQBMVW5UZ7Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872276428 (6h22m38.244855823s) [noise=true , address=45.91.116.10:4081 , piece_id=YDEAU72ZQKBHGX4R2BOOLJ6EDRCAJGZ7JLJV76A3QA3P7E537XOQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872562415 (6h22m38.24514177s) [noise=true , address=45.91.116.9:4261 , piece_id=WUVMOJPZX7L2YJP3DHMOGYO2RESZ6PL7XLP3E5PT2WHM5XBPDTXQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872861657 (6h22m38.245441012s) [noise=true , address=45.91.116.6:4975 , piece_id=ZWUNYTCKQXERWQTQJCOB4VV65JROS3CCHODQ5NZ77P5NK7P22UOQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873164256 (6h22m38.245743611s) [noise=true , address=77.90.31.17:4772 , piece_id=YMOC4YRTBQGXIPSBQRO5NGIIVRIZOB6GAUHRBSEYNUEYC3PEC6AQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873213661 (6h22m38.245793026s) [noise=true , address=45.91.116.4:4937 , piece_id=PLHQNIOBAOMV7LGCSZG4WI4WNVOJFIIZN634ISJEK2TAA2OYHJ5Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873502404 (6h22m38.246081759s) [noise=true , address=77.90.31.15:4698 , piece_id=FMKNJIQXLX4DJBMLA22ZHYXNL4XJS6GL5XFPO4I2MCSADEUSEKUQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873787228 (6h22m38.246366583s) [noise=true , address=45.91.116.5:4299 , piece_id=KMRMQCO7DEUVM6O3U5SKWRWU53AYEPKVAV5DGF5OPMUPSDR5UFSQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873831894 (6h22m38.246411249s) [noise=true , address=77.90.31.18:4806 , piece_id=C6PH5INGJYAGWOMZXHD4YCIEBKJ3ENPPSDU2TRI4YXUUJZBNGNLA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874090849 (6h22m38.246670204s) [noise=true , address=104.36.82.4:4801 , piece_id=H4RIRSEKFP2PQSASUHXTBXGYGO6V25VQSJTM5DAAXQPT2VULJWHA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874289600 (6h22m38.246868955s) [noise=true , address=45.91.116.8:4045 , piece_id=MVFBXMEJAW4GTWCHM2JLDGH53PS7W7ITLPQOGI7RJ3OWONAWI66Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874332071 (6h22m38.246911426s) [noise=true , address=45.91.116.13:4155 , piece_id=OOHUAZ76KC472TE56LLLOKNBTPX7MXO54GUICIIFU5COLN22NFBA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874767705 (6h22m38.24734706s) [noise=true , address=45.91.116.6:4974 , piece_id=3EQ7MFBZLAOV3YYUZPDZRVEGQN6RXIJSTEL2AQOBW6ZM3XMD7IYA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.875296988 (6h22m38.247876343s) [noise=true , address=104.36.82.5:4840 , piece_id=WTZIC6PVZX3GFGQ3TAI5MS6ZONYRQOHL5B4GTYDGOM337RP5AJUA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.875854935 (6h22m38.24843429s) [noise=true , address=77.90.31.16:4735 , piece_id=XIIFNPKJQXW2BE3C3WNVVCSKTFK6N2HP7YTYQMRV5KB6VZITTTXQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.876996680 (6h22m38.249576035s) [noise=true , address=77.90.31.14:4661 , piece_id=HF4PBASCVUG37QJ647CJCYC567ECPWV2QUFC2QVVO46XJXZ4HTCA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:33.562404160 (6h22m39.934983515s) [upload=5064]: Next returned nil segment Jul 19 05:13:33.562420322 (6h22m39.934999677s) [upload=5064]: Waiting for error group managing segments... Jul 19 05:13:33.913312323 (6h22m40.285891678s) [noise=true , address=45.91.116.9:4261 , piece_id=WUVMOJPZX7L2YJP3DHMOGYO2RESZ6PL7XLP3E5PT2WHM5XBPDTXQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.919093626 (6h22m40.291672981s) [noise=true , address=45.91.116.6:4975 , piece_id=ZWUNYTCKQXERWQTQJCOB4VV65JROS3CCHODQ5NZ77P5NK7P22UOQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.927887905 (6h22m40.30046726s) [noise=true , address=45.91.116.5:4299 , piece_id=KMRMQCO7DEUVM6O3U5SKWRWU53AYEPKVAV5DGF5OPMUPSDR5UFSQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.931433800 (6h22m40.304013155s) [noise=true , address=45.91.116.10:4081 , piece_id=YDEAU72ZQKBHGX4R2BOOLJ6EDRCAJGZ7JLJV76A3QA3P7E537XOQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.935178076 (6h22m40.307757431s) [noise=true , address=45.91.116.6:4974 , piece_id=3EQ7MFBZLAOV3YYUZPDZRVEGQN6RXIJSTEL2AQOBW6ZM3XMD7IYA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.945748795 (6h22m40.31832815s) [noise=true , address=45.91.116.8:4045 , piece_id=MVFBXMEJAW4GTWCHM2JLDGH53PS7W7ITLPQOGI7RJ3OWONAWI66Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.947812284 (6h22m40.320391639s) [noise=true , address=45.91.116.4:4937 , piece_id=PLHQNIOBAOMV7LGCSZG4WI4WNVOJFIIZN634ISJEK2TAA2OYHJ5Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.080127952 (6h22m40.452707307s) [noise=true , address=45.91.116.13:4155 , piece_id=OOHUAZ76KC472TE56LLLOKNBTPX7MXO54GUICIIFU5COLN22NFBA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.082690075 (6h22m40.45526943s) [noise=true , address=104.36.82.5:4840 , piece_id=WTZIC6PVZX3GFGQ3TAI5MS6ZONYRQOHL5B4GTYDGOM337RP5AJUA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.087918400 (6h22m40.460497755s) [noise=true , address=104.36.82.4:4801 , piece_id=H4RIRSEKFP2PQSASUHXTBXGYGO6V25VQSJTM5DAAXQPT2VULJWHA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:51.594432847 (6h22m57.967012202s) [noise=true , address=77.90.31.15:4698 , piece_id=FMKNJIQXLX4DJBMLA22ZHYXNL4XJS6GL5XFPO4I2MCSADEUSEKUQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:55.163396647 (6h23m1.535976002s) [noise=true , address=77.90.31.14:4661 , piece_id=HF4PBASCVUG37QJ647CJCYC567ECPWV2QUFC2QVVO46XJXZ4HTCA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:14:00.966255878 (6h23m7.338835233s) [noise=true , address=77.90.31.18:4806 , piece_id=C6PH5INGJYAGWOMZXHD4YCIEBKJ3ENPPSDU2TRI4YXUUJZBNGNLA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:14:00.966369235 (6h23m7.33894859s) [noise=true , address=77.90.31.14:4662 , piece_id=7GKZYRTV7JJFMKEE3HHZKGRBBK22UHWS6HF3MGGNPHQBMVW5UZ7Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection; write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection Jul 19 05:14:00.966401958 (6h23m7.338981313s) [noise=true , address=77.90.31.16:4735 , piece_id=XIIFNPKJQXW2BE3C3WNVVCSKTFK6N2HP7YTYQMRV5KB6VZITTTXQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection Jul 19 05:14:00.966414441 (6h23m7.338993797s) [noise=true , address=77.90.31.17:4772 , piece_id=YMOC4YRTBQGXIPSBQRO5NGIIVRIZOB6GAUHRBSEYNUEYC3PEC6AQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection Jul 19 05:14:00.966441754 (6h23m7.339021109s) [seg_pos={0 0}, upload=5064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:14:01.197829382 (6h23m7.570408747s) [upload=5064]: Done uploading segments... Jul 19 05:14:01.197859059 (6h23m7.570438424s) [upload=5064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.929843394s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 16.814865708s FAIL (2025-07-19 05:14:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 16.814888079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [......................XX...X...X....X..X....XXXXX.X.X.....XX] duration: 30.233891703s FAIL (2025-07-19 05:14:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== network connection; write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection Jul 19 05:13:30.499725691 (6h22m36.872305046s) [seg_pos={0 0}, upload=5061]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:13:30.730093449 (6h22m37.102672804s) [upload=5061]: Done uploading segments... Jul 19 05:13:30.730128235 (6h22m37.10270759s) [upload=5061]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:13:31.615702822 (6h22m37.988282217s) [upload=5064]: Starting upload Jul 19 05:13:31.615841878 (6h22m37.988421233s) [upload=5064]: Done starting upload Jul 19 05:13:31.615921711 (6h22m37.988501106s) [upload=5064]: Uploading segments... Jul 19 05:13:31.616305465 (6h22m37.98888485s) [upload=5064]: Got next segment. Inline: false Jul 19 05:13:31.616375579 (6h22m37.988954964s) [upload=5064]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:13:31.616384025 (6h22m37.988963421s) [upload=5064]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:13:31.872093568 (6h22m38.244672923s) [seg_pos={0 0}, upload=5064]: Begin upload segment... Jul 19 05:13:31.872131390 (6h22m38.244710735s) [seg_pos={0 0}, upload=5064]: Done begin upload segment. Jul 19 05:13:31.872182527 (6h22m38.244761883s) [noise=true , address=77.90.31.14:4662 , piece_id=7GKZYRTV7JJFMKEE3HHZKGRBBK22UHWS6HF3MGGNPHQBMVW5UZ7Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872276428 (6h22m38.244855823s) [noise=true , address=45.91.116.10:4081 , piece_id=YDEAU72ZQKBHGX4R2BOOLJ6EDRCAJGZ7JLJV76A3QA3P7E537XOQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872562415 (6h22m38.24514177s) [noise=true , address=45.91.116.9:4261 , piece_id=WUVMOJPZX7L2YJP3DHMOGYO2RESZ6PL7XLP3E5PT2WHM5XBPDTXQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.872861657 (6h22m38.245441012s) [noise=true , address=45.91.116.6:4975 , piece_id=ZWUNYTCKQXERWQTQJCOB4VV65JROS3CCHODQ5NZ77P5NK7P22UOQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873164256 (6h22m38.245743611s) [noise=true , address=77.90.31.17:4772 , piece_id=YMOC4YRTBQGXIPSBQRO5NGIIVRIZOB6GAUHRBSEYNUEYC3PEC6AQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873213661 (6h22m38.245793026s) [noise=true , address=45.91.116.4:4937 , piece_id=PLHQNIOBAOMV7LGCSZG4WI4WNVOJFIIZN634ISJEK2TAA2OYHJ5Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873502404 (6h22m38.246081759s) [noise=true , address=77.90.31.15:4698 , piece_id=FMKNJIQXLX4DJBMLA22ZHYXNL4XJS6GL5XFPO4I2MCSADEUSEKUQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873787228 (6h22m38.246366583s) [noise=true , address=45.91.116.5:4299 , piece_id=KMRMQCO7DEUVM6O3U5SKWRWU53AYEPKVAV5DGF5OPMUPSDR5UFSQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.873831894 (6h22m38.246411249s) [noise=true , address=77.90.31.18:4806 , piece_id=C6PH5INGJYAGWOMZXHD4YCIEBKJ3ENPPSDU2TRI4YXUUJZBNGNLA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874090849 (6h22m38.246670204s) [noise=true , address=104.36.82.4:4801 , piece_id=H4RIRSEKFP2PQSASUHXTBXGYGO6V25VQSJTM5DAAXQPT2VULJWHA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874289600 (6h22m38.246868955s) [noise=true , address=45.91.116.8:4045 , piece_id=MVFBXMEJAW4GTWCHM2JLDGH53PS7W7ITLPQOGI7RJ3OWONAWI66Q, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874332071 (6h22m38.246911426s) [noise=true , address=45.91.116.13:4155 , piece_id=OOHUAZ76KC472TE56LLLOKNBTPX7MXO54GUICIIFU5COLN22NFBA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.874767705 (6h22m38.24734706s) [noise=true , address=45.91.116.6:4974 , piece_id=3EQ7MFBZLAOV3YYUZPDZRVEGQN6RXIJSTEL2AQOBW6ZM3XMD7IYA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.875296988 (6h22m38.247876343s) [noise=true , address=104.36.82.5:4840 , piece_id=WTZIC6PVZX3GFGQ3TAI5MS6ZONYRQOHL5B4GTYDGOM337RP5AJUA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.875854935 (6h22m38.24843429s) [noise=true , address=77.90.31.16:4735 , piece_id=XIIFNPKJQXW2BE3C3WNVVCSKTFK6N2HP7YTYQMRV5KB6VZITTTXQ, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:31.876996680 (6h22m38.249576035s) [noise=true , address=77.90.31.14:4661 , piece_id=HF4PBASCVUG37QJ647CJCYC567ECPWV2QUFC2QVVO46XJXZ4HTCA, seg_pos={0 0}, upload=5064]: Uploading piece... Jul 19 05:13:33.562404160 (6h22m39.934983515s) [upload=5064]: Next returned nil segment Jul 19 05:13:33.562420322 (6h22m39.934999677s) [upload=5064]: Waiting for error group managing segments... Jul 19 05:13:33.913312323 (6h22m40.285891678s) [noise=true , address=45.91.116.9:4261 , piece_id=WUVMOJPZX7L2YJP3DHMOGYO2RESZ6PL7XLP3E5PT2WHM5XBPDTXQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.919093626 (6h22m40.291672981s) [noise=true , address=45.91.116.6:4975 , piece_id=ZWUNYTCKQXERWQTQJCOB4VV65JROS3CCHODQ5NZ77P5NK7P22UOQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.927887905 (6h22m40.30046726s) [noise=true , address=45.91.116.5:4299 , piece_id=KMRMQCO7DEUVM6O3U5SKWRWU53AYEPKVAV5DGF5OPMUPSDR5UFSQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.931433800 (6h22m40.304013155s) [noise=true , address=45.91.116.10:4081 , piece_id=YDEAU72ZQKBHGX4R2BOOLJ6EDRCAJGZ7JLJV76A3QA3P7E537XOQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.935178076 (6h22m40.307757431s) [noise=true , address=45.91.116.6:4974 , piece_id=3EQ7MFBZLAOV3YYUZPDZRVEGQN6RXIJSTEL2AQOBW6ZM3XMD7IYA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.945748795 (6h22m40.31832815s) [noise=true , address=45.91.116.8:4045 , piece_id=MVFBXMEJAW4GTWCHM2JLDGH53PS7W7ITLPQOGI7RJ3OWONAWI66Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:33.947812284 (6h22m40.320391639s) [noise=true , address=45.91.116.4:4937 , piece_id=PLHQNIOBAOMV7LGCSZG4WI4WNVOJFIIZN634ISJEK2TAA2OYHJ5Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.080127952 (6h22m40.452707307s) [noise=true , address=45.91.116.13:4155 , piece_id=OOHUAZ76KC472TE56LLLOKNBTPX7MXO54GUICIIFU5COLN22NFBA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.082690075 (6h22m40.45526943s) [noise=true , address=104.36.82.5:4840 , piece_id=WTZIC6PVZX3GFGQ3TAI5MS6ZONYRQOHL5B4GTYDGOM337RP5AJUA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:34.087918400 (6h22m40.460497755s) [noise=true , address=104.36.82.4:4801 , piece_id=H4RIRSEKFP2PQSASUHXTBXGYGO6V25VQSJTM5DAAXQPT2VULJWHA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:51.594432847 (6h22m57.967012202s) [noise=true , address=77.90.31.15:4698 , piece_id=FMKNJIQXLX4DJBMLA22ZHYXNL4XJS6GL5XFPO4I2MCSADEUSEKUQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:13:55.163396647 (6h23m1.535976002s) [noise=true , address=77.90.31.14:4661 , piece_id=HF4PBASCVUG37QJ647CJCYC567ECPWV2QUFC2QVVO46XJXZ4HTCA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: <nil> Jul 19 05:14:00.966255878 (6h23m7.338835233s) [noise=true , address=77.90.31.18:4806 , piece_id=C6PH5INGJYAGWOMZXHD4YCIEBKJ3ENPPSDU2TRI4YXUUJZBNGNLA, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:14:00.966369235 (6h23m7.33894859s) [noise=true , address=77.90.31.14:4662 , piece_id=7GKZYRTV7JJFMKEE3HHZKGRBBK22UHWS6HF3MGGNPHQBMVW5UZ7Q, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection; write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56328->77.90.31.14:4662: use of closed network connection Jul 19 05:14:00.966401958 (6h23m7.338981313s) [noise=true , address=77.90.31.16:4735 , piece_id=XIIFNPKJQXW2BE3C3WNVVCSKTFK6N2HP7YTYQMRV5KB6VZITTTXQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58392->77.90.31.16:4735: use of closed network connection Jul 19 05:14:00.966414441 (6h23m7.338993797s) [noise=true , address=77.90.31.17:4772 , piece_id=YMOC4YRTBQGXIPSBQRO5NGIIVRIZOB6GAUHRBSEYNUEYC3PEC6AQ, seg_pos={0 0}, upload=5064]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection; write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50432->77.90.31.17:4772: use of closed network connection Jul 19 05:14:00.966441754 (6h23m7.339021109s) [seg_pos={0 0}, upload=5064]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:14:01.197829382 (6h23m7.570408747s) [upload=5064]: Done uploading segments... Jul 19 05:14:01.197859059 (6h23m7.570438424s) [upload=5064]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.233906382s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......................XX...X...X....X..X....XXXXX.X.X.....X] duration: 30.231991163s FAIL (2025-07-19 05:13:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 8:4805 , piece_id=WNU3XPTJXHVMLS6VC5EO6K36RMDYVIU35J6EQ2TVRUE4JPZP3PPA, seg_pos={0 0}, upload=5046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 05:12:11.346253802 (6h21m17.718833157s) [seg_pos={0 0}, upload=5046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:12:11.346346349 (6h21m17.718925714s) [upload=5046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:12:11.346352841 (6h21m17.718932196s) [upload=5046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:12:11.658573508 (6h21m18.031152863s) [upload=5046]: Done uploading segments... Jul 19 05:12:11.658594047 (6h21m18.031173402s) [upload=5046]: Upload finished. err: <nil> Jul 19 05:13:01.140644889 (6h22m7.513224244s) [upload=5061]: Starting upload Jul 19 05:13:01.140727768 (6h22m7.513307123s) [upload=5061]: Done starting upload Jul 19 05:13:01.140851815 (6h22m7.51343117s) [upload=5061]: Uploading segments... Jul 19 05:13:01.140949472 (6h22m7.513528827s) [upload=5061]: Got next segment. Inline: false Jul 19 05:13:01.140984889 (6h22m7.513564244s) [upload=5061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:13:01.140997755 (6h22m7.51357711s) [upload=5061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:13:01.393965340 (6h22m7.766544695s) [seg_pos={0 0}, upload=5061]: Begin upload segment... Jul 19 05:13:01.394009123 (6h22m7.766588478s) [seg_pos={0 0}, upload=5061]: Done begin upload segment. Jul 19 05:13:01.394033981 (6h22m7.766613336s) [noise=true , address=104.36.82.15:4258 , piece_id=K2MYX5D4J6HMHBWJDU6RYKUQEWRZ5EMRX65P7IOLFWI42LI5CIGA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.394103294 (6h22m7.766682659s) [noise=true , address=45.91.116.7:4010 , piece_id=ZQUQBEZ2MYZ76MQROXFMDDT7XRQIRZ6GI6SDFNIQAPRBZV6AB7OA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.394446921 (6h22m7.767026276s) [noise=true , address=45.91.116.12:4226 , piece_id=DY2H2SJOH6BMVAKSRM2HDQCERKJ7BWTW6JXNONU5QMOXKQXXT6MA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.394633318 (6h22m7.767212673s) [noise=true , address=77.90.31.11:4554 , piece_id=ZYOAIZOTTSX2YFVHNSTFO2WJN62NU2ZZMHTO37ZT3X3J36HQE5RA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.394920708 (6h22m7.767500063s) [noise=true , address=77.90.31.17:4771 , piece_id=MUDT6RGUD2ZRHJWCY5OSRAMV5D5AP4MKIZKTFMRXDHSG3YNYPN3Q, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395002735 (6h22m7.76758209s) [noise=true , address=45.91.116.4:4940 , piece_id=GMKBHSP3J3Q52HKK5MAT474IPPLUG3WMPEP7LQ4KM2KB5XZY4CFQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395292549 (6h22m7.767871914s) [noise=true , address=45.91.116.12:4227 , piece_id=PTMQUYSFEROMWZHGAIRDMVF3VOUHYNWRTC7NEAHAG5YO6ATGIFJA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395653711 (6h22m7.768233066s) [noise=true , address=45.91.116.14:4191 , piece_id=QUNQC5RP4FD37YMTP7PYHAL6SSYEJZJMGWE5TGN2THFYKEDNAVJQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395676564 (6h22m7.768255919s) [noise=true , address=77.90.31.14:4661 , piece_id=LNPTMD7K3DDMX5N7HM5H4IL4UFAY4N3ORTSZEOGG4C4IVTT6EUJQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395895213 (6h22m7.768474568s) [noise=true , address=45.91.116.6:4974 , piece_id=3P5UOXZ2JCBDUR7UQ7SXIJAABU26NF5EWNEL2E77UC5SMI5OYCCQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.395960017 (6h22m7.768539372s) [noise=true , address=45.91.116.9:4263 , piece_id=BXWDBJIAXEYU2P6POCAB52HI7KPA6HA5SAQNQS2E4CRNLS2LOJ4Q, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.396225696 (6h22m7.768805051s) [noise=true , address=77.90.31.10:4520 , piece_id=7DIMJKGHXPP3DMQDFODZDJRGCOVHR26OXFMHRQ4JFR3MBN34D5DQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.396265582 (6h22m7.768844937s) [noise=true , address=104.36.82.11:4114 , piece_id=MFXVM2QAOQFAFPIEXBHAF7S6WA4TM5MR7DQUEAYJFTVCVCM3E4GQ, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.396494310 (6h22m7.769073665s) [noise=true , address=45.91.116.10:4081 , piece_id=F5CLAYNS7J3DODS6XSEUXPZXXLU22EJ7SEFMJ6JE2GHMJ2MZHI6A, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.396567389 (6h22m7.769146744s) [noise=true , address=45.91.116.7:4011 , piece_id=KORMU6LCNH3TU64FG7JGZB7KRYZ33XK6GACS5IBWK2MXIU4XLPRA, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:01.396754327 (6h22m7.769333682s) [noise=true , address=45.91.116.11:4117 , piece_id=BBCY4YWMDEX25J3M466YPZ33C7OKZOT77MDTB6I62VCFHUN3KX6A, seg_pos={0 0}, upload=5061]: Uploading piece... Jul 19 05:13:03.153545360 (6h22m9.526124715s) [upload=5061]: Next returned nil segment Jul 19 05:13:03.153560329 (6h22m9.526139674s) [upload=5061]: Waiting for error group managing segments... Jul 19 05:13:03.443194182 (6h22m9.815773537s) [noise=true , address=45.91.116.9:4263 , piece_id=BXWDBJIAXEYU2P6POCAB52HI7KPA6HA5SAQNQS2E4CRNLS2LOJ4Q, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.450560849 (6h22m9.823140204s) [noise=true , address=45.91.116.4:4940 , piece_id=GMKBHSP3J3Q52HKK5MAT474IPPLUG3WMPEP7LQ4KM2KB5XZY4CFQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.453309278 (6h22m9.825888633s) [noise=true , address=45.91.116.12:4226 , piece_id=DY2H2SJOH6BMVAKSRM2HDQCERKJ7BWTW6JXNONU5QMOXKQXXT6MA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.460095565 (6h22m9.83267492s) [noise=true , address=45.91.116.6:4974 , piece_id=3P5UOXZ2JCBDUR7UQ7SXIJAABU26NF5EWNEL2E77UC5SMI5OYCCQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.468362225 (6h22m9.84094158s) [noise=true , address=45.91.116.10:4081 , piece_id=F5CLAYNS7J3DODS6XSEUXPZXXLU22EJ7SEFMJ6JE2GHMJ2MZHI6A, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.470190894 (6h22m9.84277025s) [noise=true , address=45.91.116.7:4011 , piece_id=KORMU6LCNH3TU64FG7JGZB7KRYZ33XK6GACS5IBWK2MXIU4XLPRA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.546087355 (6h22m9.91866671s) [noise=true , address=45.91.116.12:4227 , piece_id=PTMQUYSFEROMWZHGAIRDMVF3VOUHYNWRTC7NEAHAG5YO6ATGIFJA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.553001787 (6h22m9.925581142s) [noise=true , address=45.91.116.7:4010 , piece_id=ZQUQBEZ2MYZ76MQROXFMDDT7XRQIRZ6GI6SDFNIQAPRBZV6AB7OA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.586073515 (6h22m9.95865287s) [noise=true , address=104.36.82.11:4114 , piece_id=MFXVM2QAOQFAFPIEXBHAF7S6WA4TM5MR7DQUEAYJFTVCVCM3E4GQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.608633908 (6h22m9.981213263s) [noise=true , address=45.91.116.11:4117 , piece_id=BBCY4YWMDEX25J3M466YPZ33C7OKZOT77MDTB6I62VCFHUN3KX6A, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.613407333 (6h22m9.985986698s) [noise=true , address=45.91.116.14:4191 , piece_id=QUNQC5RP4FD37YMTP7PYHAL6SSYEJZJMGWE5TGN2THFYKEDNAVJQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:03.621088622 (6h22m9.993667977s) [noise=true , address=104.36.82.15:4258 , piece_id=K2MYX5D4J6HMHBWJDU6RYKUQEWRZ5EMRX65P7IOLFWI42LI5CIGA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: <nil> Jul 19 05:13:30.499554093 (6h22m36.872133438s) [noise=true , address=77.90.31.14:4661 , piece_id=LNPTMD7K3DDMX5N7HM5H4IL4UFAY4N3ORTSZEOGG4C4IVTT6EUJQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:13:30.499588759 (6h22m36.872168114s) [noise=true , address=77.90.31.10:4520 , piece_id=7DIMJKGHXPP3DMQDFODZDJRGCOVHR26OXFMHRQ4JFR3MBN34D5DQ, seg_pos={0 0}, upload=5061]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:13:30.499668452 (6h22m36.872247807s) [noise=true , address=77.90.31.11:4554 , piece_id=ZYOAIZOTTSX2YFVHNSTFO2WJN62NU2ZZMHTO37ZT3X3J36HQE5RA, seg_pos={0 0}, upload=5061]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:42192->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:42192->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42192->77.90.31.11:4554: use of closed network connection Jul 19 05:13:30.499708157 (6h22m36.872287512s) [noise=true , address=77.90.31.17:4771 , piece_id=MUDT6RGUD2ZRHJWCY5OSRAMV5D5AP4MKIZKTFMRXDHSG3YNYPN3Q, seg_pos={0 0}, upload=5061]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39440->77.90.31.17:4771: use of closed network connection Jul 19 05:13:30.499725691 (6h22m36.872305046s) [seg_pos={0 0}, upload=5061]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:13:30.730093449 (6h22m37.102672804s) [upload=5061]: Done uploading segments... Jul 19 05:13:30.730128235 (6h22m37.10270759s) [upload=5061]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.232003126s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 9.430533751s FAIL (2025-07-19 05:12:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 9.430565483s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 8.368369355s FAIL (2025-07-19 05:12:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 8.368400405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........................XX...X...X....X..X....XXXXX.X.X.....] duration: 7.879249541s FAIL (2025-07-19 05:12:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.87927533s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................XX...X...X....X..X....XXXXX.X.X....] duration: 5.780683075s FAIL (2025-07-19 05:11:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.780713294s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.508858139s FAIL (2025-07-19 05:11:28 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.508886104s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........................XX...X...X....X..X....XXXXX.X.X...] duration: 5.800191955s FAIL (2025-07-19 05:11:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.800217705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.123818105s FAIL (2025-07-19 05:10:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 7.123836702s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...........................XX...X...X....X..X....XXXXX.X.X..] duration: 15.087940088s FAIL (2025-07-19 05:10:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 15.087967561s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.946452522s FAIL (2025-07-19 05:09:39 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.946481658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [............................XX...X...X....X..X....XXXXX.X.X.] duration: 5.899609796s FAIL (2025-07-19 05:09:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.899642679s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 27.567721207s FAIL (2025-07-19 05:09:19 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 27.567740253s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.............................XX...X...X....X..X....XXXXX.X.X] duration: 30.233523858s FAIL (2025-07-19 05:09:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== : <nil> Jul 19 05:08:29.800544823 (6h17m36.173124169s) [upload=4997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:08:29.800548410 (6h17m36.173127765s) [upload=4997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:08:30.148027600 (6h17m36.520606955s) [upload=4997]: Done uploading segments... Jul 19 05:08:30.148148401 (6h17m36.520727746s) [upload=4997]: Upload finished. err: <nil> Jul 19 05:08:48.807983386 (6h17m55.180562741s) [upload=5005]: Starting upload Jul 19 05:08:48.808044453 (6h17m55.180623808s) [upload=5005]: Done starting upload Jul 19 05:08:48.808175243 (6h17m55.180754598s) [upload=5005]: Uploading segments... Jul 19 05:08:48.808334198 (6h17m55.180913553s) [upload=5005]: Got next segment. Inline: false Jul 19 05:08:48.808392509 (6h17m55.180971874s) [upload=5005]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:08:48.808399823 (6h17m55.180979188s) [upload=5005]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:08:49.075515233 (6h17m55.448094578s) [seg_pos={0 0}, upload=5005]: Begin upload segment... Jul 19 05:08:49.075564547 (6h17m55.448143902s) [seg_pos={0 0}, upload=5005]: Done begin upload segment. Jul 19 05:08:49.075602881 (6h17m55.448182236s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075612279 (6h17m55.448191634s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.075964213 (6h17m55.448543568s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076122025 (6h17m55.44870137s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076272903 (6h17m55.448852258s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076533412 (6h17m55.449112767s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.076794962 (6h17m55.449374317s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077044971 (6h17m55.449624326s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077296953 (6h17m55.449876308s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077501123 (6h17m55.450080468s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077695656 (6h17m55.450275001s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.077994558 (6h17m55.450573913s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078213187 (6h17m55.450792542s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.078412077 (6h17m55.450991432s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.079649705 (6h17m55.45222906s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:49.080903344 (6h17m55.453482699s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Uploading piece... Jul 19 05:08:50.857952826 (6h17m57.230532181s) [upload=5005]: Next returned nil segment Jul 19 05:08:50.857969477 (6h17m57.230548832s) [upload=5005]: Waiting for error group managing segments... Jul 19 05:08:51.139405061 (6h17m57.511984416s) [noise=true , address=104.36.82.13:4186 , piece_id=PPK6XEQYJ3XV27XUJYKUL2MYAR5KUYKF5AHJS5EBXOK7PZWWM66Q, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.143987831 (6h17m57.516567176s) [noise=true , address=45.91.116.8:4048 , piece_id=QKXGR6YZSYTQIUXV54DRYQNWY352UFS2Z2VXWFJTD7YJYJZ24NCQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.148258443 (6h17m57.520837798s) [noise=true , address=104.36.82.16:4293 , piece_id=2NA7I27WVAWAUWDCZVYBJLNXGWIDNJMKZU7YFJZSXCQMCHKLCGXA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.156796512 (6h17m57.529375867s) [noise=true , address=45.91.116.13:4154 , piece_id=OJO7VBXGBLV2SG2AQ6BPBP57SL75WV4UZSWC3O5HXD7ROBICSYGA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.193908742 (6h17m57.566488098s) [noise=true , address=104.36.82.14:4222 , piece_id=4F6ZDS2KMQNALT2YV6D3TAZLIBFUFBM5NNCBGLSE6L6PR5QMGPVA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.217498736 (6h17m57.590078091s) [noise=true , address=45.91.116.4:4939 , piece_id=J2BIIQITH7R54K4ENE6AKU7KCOUAGYBOTKX6VF6FY7FNQDMNNFCA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.218585386 (6h17m57.591164741s) [noise=true , address=45.91.116.9:4263 , piece_id=4L3UWFKV3AC7F4RPS7GJFXOO6DJNCPMFB5NWGJH6HXAFBGLYAMYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.290646027 (6h17m57.663225392s) [noise=true , address=104.36.82.2:4732 , piece_id=DBPJH5FSNFRPIBLP7HWWJFVVXYC47J7UGPCXFL7L3BD6BGFSJDFA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.309329961 (6h17m57.681909316s) [noise=true , address=104.36.82.12:4150 , piece_id=GQ4GQQZHK3Y4KVVB2IUU4LEEBR4OJ2YSS5Z5DF7WHHDCYYSSGRYA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:51.317895332 (6h17m57.690474687s) [noise=true , address=104.36.82.9:4007 , piece_id=R2IPGORJ3Z4FFCC52P5WMXYIA6O5V7DDZHHL4X64XV6J36SIWSBQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:08:52.525479047 (6h17m58.898058402s) [noise=true , address=45.91.116.9:4262 , piece_id=EXWTIYPESLJQK3CDK732WC63OZ7OJOQAUCBPGO3ZU54UBFUJ3TSA, seg_pos={0 0}, upload=5005]: Done uploading piece. err: <nil> Jul 19 05:09:18.157484198 (6h18m24.530063553s) [noise=true , address=77.90.31.13:4628 , piece_id=OAGGFQO2CORAZQV6ND3EERSICEOSHKK73Y3PUWLT45ZUWEWDHOFQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157617684 (6h18m24.530197039s) [noise=true , address=77.90.31.15:4698 , piece_id=HNTVTQMVXDUMK4CMHKVUZ7DHM3MZL56TGXFR4FFFV2V3DCTLAVZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:09:18.157744796 (6h18m24.530324151s) [noise=true , address=77.90.31.11:4555 , piece_id=QGRXVFSKXDCLKSSSXEK4QYZEETWQGFZMQQBSTXNMAHY45RA6MAVQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38288->77.90.31.11:4555: use of closed network connection Jul 19 05:09:18.157785745 (6h18m24.5303651s) [noise=true , address=77.90.31.13:4626 , piece_id=2QAZAYJWKJXTPY475RLOSPV3ZZTMB6OZLNYAKD7FCFV7T4WPC5ZQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58846->77.90.31.13:4626: use of closed network connection Jul 19 05:09:18.157859036 (6h18m24.530438391s) [noise=true , address=77.90.31.12:4592 , piece_id=HS5YQKPTPUVHFZH3VZ7X2LAO2SVS6JZOTHTLJ6T47G3CLTZTR7UQ, seg_pos={0 0}, upload=5005]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38220->77.90.31.12:4592: use of closed network connection Jul 19 05:09:18.157898821 (6h18m24.530478176s) [seg_pos={0 0}, upload=5005]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 05:09:18.389974737 (6h18m24.762554092s) [upload=5005]: Done uploading segments... Jul 19 05:09:18.389994916 (6h18m24.762574271s) [upload=5005]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.233549267s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..............................XX...X...X....X..X....XXXXX.X.] duration: 22.433473572s FAIL (2025-07-19 05:08:32 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=4978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: read tcp 188.166.225.170:56432->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 05:06:53.739102075 (6h16m0.11168143s) [seg_pos={0 0}, upload=4978]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:06:53.739197288 (6h16m0.111776673s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:06:53.739203930 (6h16m0.111783286s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:06:54.046241769 (6h16m0.418821124s) [upload=4978]: Done uploading segments... Jul 19 05:06:54.046260585 (6h16m0.41883994s) [upload=4978]: Upload finished. err: <nil> Jul 19 05:07:18.967786786 (6h16m25.340366131s) [upload=4987]: Starting upload Jul 19 05:07:18.967842954 (6h16m25.340422309s) [upload=4987]: Done starting upload Jul 19 05:07:18.967928819 (6h16m25.340508174s) [upload=4987]: Uploading segments... Jul 19 05:07:18.968231738 (6h16m25.340811094s) [upload=4987]: Got next segment. Inline: false Jul 19 05:07:18.968276884 (6h16m25.340856239s) [upload=4987]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:07:18.968282466 (6h16m25.340861821s) [upload=4987]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:07:19.221977372 (6h16m25.594556728s) [seg_pos={0 0}, upload=4987]: Begin upload segment... Jul 19 05:07:19.222023461 (6h16m25.594602816s) [seg_pos={0 0}, upload=4987]: Done begin upload segment. Jul 19 05:07:19.222042989 (6h16m25.594622334s) [noise=true , address=77.90.31.15:4699 , piece_id=AGUDMA4LGNU3DSZB42QLS7M4JEZ6N5O7PNOAUOMLIQJEGISI3W5Q, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222313176 (6h16m25.594892531s) [noise=true , address=77.90.31.15:4697 , piece_id=544MNCQQNU6A75WX2WIJQ6A3F4N4GCCSSX3C4UTT7WHH77QOCUHA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222568183 (6h16m25.595147538s) [noise=true , address=104.36.82.4:4801 , piece_id=3O2Q25TEDWVGN5PKMUMVK2TGYGXKRDWPV3ZY32JL7HQCDO3FB6KA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222804205 (6h16m25.59538356s) [noise=true , address=77.90.31.14:4663 , piece_id=6XQO4MRE2VYQHQAQR3BGOLYFPFDR3GNOQNO4NHNZJCQKGWVO6JAQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223029146 (6h16m25.595608501s) [noise=true , address=77.90.31.10:4520 , piece_id=FOH2YFXGUQ772SNJXDP7ZR7CF5LFZ3LGYKCIEOEM4LJHPSMRMYHQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223246592 (6h16m25.595825947s) [noise=true , address=45.91.116.6:4974 , piece_id=U3Z2FOR3EZI3P2MCADPDJL6JKKUSQODAYCD263AM22E2WMRIFX4A, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223512381 (6h16m25.596091736s) [noise=true , address=45.91.116.6:4973 , piece_id=HWJBA2OCLJKSN64HSRK7RULJHIQPVKIT3GWPK64Y32VUUYFEUTMQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223703747 (6h16m25.596283102s) [noise=true , address=77.90.31.17:4772 , piece_id=VP3JS4QYDXJ6R35MFRUK2XULQLGJJ23BSMM7IRWMVR3IISVEBLUQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223956139 (6h16m25.596535495s) [noise=true , address=77.90.31.12:4592 , piece_id=36WYINRSYTUIPPO76JUD7HAO6SX4NW25G37XPOU3YVT2QLACABIA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224198243 (6h16m25.596777598s) [noise=true , address=104.36.82.5:4837 , piece_id=7YZBYKPWKCGARRJLE7WK3YRR536QC7BZZFUHYGCGQRTHRX3PTSPQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224392975 (6h16m25.59697233s) [noise=true , address=77.90.31.21:4878 , piece_id=PSLYCIRGFJ6RTL65HOEO3S2JDC737V5MJ6R3PGJJ3ODMDPNIHWCA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224586385 (6h16m25.59716574s) [noise=true , address=77.90.31.18:4807 , piece_id=3S6MGZJ2IC7UVNLPWKJCPBCY23B2DLGIJ6ANTNPNB4UKDIW45Y7Q, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224772702 (6h16m25.597352057s) [noise=true , address=45.91.116.13:4154 , piece_id=JFXGESP7N4C4ZQ7TWKTOQVVKQ3APV6Y3C6NBN7OIKXYJMJ5RPHWA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225006720 (6h16m25.597586075s) [noise=true , address=104.36.82.7:4909 , piece_id=YCN3RDTWQZLZNYCVOQ334JY2XZJFAXDXZ5CZBWOTN4I2XI5U3FMA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225222964 (6h16m25.597802319s) [noise=true , address=45.91.116.9:4261 , piece_id=VGBFN7ZKPHS5S4SDWUHEKY6F7LS6JTVRY3ITUAGXHYWNWQMLJOIA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225416364 (6h16m25.597995719s) [noise=true , address=45.91.116.4:4939 , piece_id=BLQFFPUNAGDF7FCF2JPLBPPUAEWI7FMG4DKD7TWRMPQTYX3LEUAQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:20.980351225 (6h16m27.35293058s) [upload=4987]: Next returned nil segment Jul 19 05:07:20.980380271 (6h16m27.352959626s) [upload=4987]: Waiting for error group managing segments... Jul 19 05:07:21.267793974 (6h16m27.640373329s) [noise=true , address=104.36.82.4:4801 , piece_id=3O2Q25TEDWVGN5PKMUMVK2TGYGXKRDWPV3ZY32JL7HQCDO3FB6KA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.285583227 (6h16m27.658162583s) [noise=true , address=45.91.116.13:4154 , piece_id=JFXGESP7N4C4ZQ7TWKTOQVVKQ3APV6Y3C6NBN7OIKXYJMJ5RPHWA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.286980110 (6h16m27.659559465s) [noise=true , address=45.91.116.9:4261 , piece_id=VGBFN7ZKPHS5S4SDWUHEKY6F7LS6JTVRY3ITUAGXHYWNWQMLJOIA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.359982162 (6h16m27.732561517s) [noise=true , address=104.36.82.5:4837 , piece_id=7YZBYKPWKCGARRJLE7WK3YRR536QC7BZZFUHYGCGQRTHRX3PTSPQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.363819385 (6h16m27.73639874s) [noise=true , address=45.91.116.6:4974 , piece_id=U3Z2FOR3EZI3P2MCADPDJL6JKKUSQODAYCD263AM22E2WMRIFX4A, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.372194202 (6h16m27.744773557s) [noise=true , address=45.91.116.6:4973 , piece_id=HWJBA2OCLJKSN64HSRK7RULJHIQPVKIT3GWPK64Y32VUUYFEUTMQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.375296359 (6h16m27.747875714s) [noise=true , address=104.36.82.7:4909 , piece_id=YCN3RDTWQZLZNYCVOQ334JY2XZJFAXDXZ5CZBWOTN4I2XI5U3FMA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.381291291 (6h16m27.753870646s) [noise=true , address=45.91.116.4:4939 , piece_id=BLQFFPUNAGDF7FCF2JPLBPPUAEWI7FMG4DKD7TWRMPQTYX3LEUAQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:32.446172843 (6h16m38.818752198s) [noise=true , address=77.90.31.17:4772 , piece_id=VP3JS4QYDXJ6R35MFRUK2XULQLGJJ23BSMM7IRWMVR3IISVEBLUQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:37.950304448 (6h16m44.322883803s) [noise=true , address=77.90.31.10:4520 , piece_id=FOH2YFXGUQ772SNJXDP7ZR7CF5LFZ3LGYKCIEOEM4LJHPSMRMYHQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:38.051458593 (6h16m44.424037949s) [noise=true , address=77.90.31.15:4697 , piece_id=544MNCQQNU6A75WX2WIJQ6A3F4N4GCCSSX3C4UTT7WHH77QOCUHA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:38.236231036 (6h16m44.608810401s) [noise=true , address=77.90.31.14:4663 , piece_id=6XQO4MRE2VYQHQAQR3BGOLYFPFDR3GNOQNO4NHNZJCQKGWVO6JAQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:48.143474611 (6h16m54.516053967s) [noise=true , address=77.90.31.21:4878 , piece_id=PSLYCIRGFJ6RTL65HOEO3S2JDC737V5MJ6R3PGJJ3ODMDPNIHWCA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:48.143499839 (6h16m54.516079194s) [seg_pos={0 0}, upload=4987]: Segment reached optimal threshold of 13 pieces. Jul 19 05:07:48.143654295 (6h16m54.51623364s) [noise=true , address=77.90.31.18:4807 , piece_id=3S6MGZJ2IC7UVNLPWKJCPBCY23B2DLGIJ6ANTNPNB4UKDIW45Y7Q, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Jul 19 05:07:48.143694843 (6h16m54.516274198s) [noise=true , address=77.90.31.12:4592 , piece_id=36WYINRSYTUIPPO76JUD7HAO6SX4NW25G37XPOU3YVT2QLACABIA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 19 05:07:48.143734959 (6h16m54.516314314s) [noise=true , address=77.90.31.15:4699 , piece_id=AGUDMA4LGNU3DSZB42QLS7M4JEZ6N5O7PNOAUOMLIQJEGISI3W5Q, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection Jul 19 05:07:48.143753064 (6h16m54.516332419s) [seg_pos={0 0}, upload=4987]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:07:48.143813039 (6h16m54.516392384s) [upload=4987]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:07:48.143816736 (6h16m54.516396091s) [upload=4987]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:07:48.532152020 (6h16m54.904731375s) [upload=4987]: Done uploading segments... Jul 19 05:07:48.532172109 (6h16m54.904751464s) [upload=4987]: Upload finished. err: metaclient: context deadline exceeded Duration: 22.433497338s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 11.956095934s FAIL (2025-07-19 05:08:02 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 11.956122926s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...............................XX...X...X....X..X....XXXXX.X] duration: 30.226194444s FAIL (2025-07-19 05:07:48 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== , upload=4978]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: read tcp 188.166.225.170:56432->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 05:06:53.739102075 (6h16m0.11168143s) [seg_pos={0 0}, upload=4978]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:06:53.739197288 (6h16m0.111776673s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:06:53.739203930 (6h16m0.111783286s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:06:54.046241769 (6h16m0.418821124s) [upload=4978]: Done uploading segments... Jul 19 05:06:54.046260585 (6h16m0.41883994s) [upload=4978]: Upload finished. err: <nil> Jul 19 05:07:18.967786786 (6h16m25.340366131s) [upload=4987]: Starting upload Jul 19 05:07:18.967842954 (6h16m25.340422309s) [upload=4987]: Done starting upload Jul 19 05:07:18.967928819 (6h16m25.340508174s) [upload=4987]: Uploading segments... Jul 19 05:07:18.968231738 (6h16m25.340811094s) [upload=4987]: Got next segment. Inline: false Jul 19 05:07:18.968276884 (6h16m25.340856239s) [upload=4987]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:07:18.968282466 (6h16m25.340861821s) [upload=4987]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:07:19.221977372 (6h16m25.594556728s) [seg_pos={0 0}, upload=4987]: Begin upload segment... Jul 19 05:07:19.222023461 (6h16m25.594602816s) [seg_pos={0 0}, upload=4987]: Done begin upload segment. Jul 19 05:07:19.222042989 (6h16m25.594622334s) [noise=true , address=77.90.31.15:4699 , piece_id=AGUDMA4LGNU3DSZB42QLS7M4JEZ6N5O7PNOAUOMLIQJEGISI3W5Q, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222313176 (6h16m25.594892531s) [noise=true , address=77.90.31.15:4697 , piece_id=544MNCQQNU6A75WX2WIJQ6A3F4N4GCCSSX3C4UTT7WHH77QOCUHA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222568183 (6h16m25.595147538s) [noise=true , address=104.36.82.4:4801 , piece_id=3O2Q25TEDWVGN5PKMUMVK2TGYGXKRDWPV3ZY32JL7HQCDO3FB6KA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.222804205 (6h16m25.59538356s) [noise=true , address=77.90.31.14:4663 , piece_id=6XQO4MRE2VYQHQAQR3BGOLYFPFDR3GNOQNO4NHNZJCQKGWVO6JAQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223029146 (6h16m25.595608501s) [noise=true , address=77.90.31.10:4520 , piece_id=FOH2YFXGUQ772SNJXDP7ZR7CF5LFZ3LGYKCIEOEM4LJHPSMRMYHQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223246592 (6h16m25.595825947s) [noise=true , address=45.91.116.6:4974 , piece_id=U3Z2FOR3EZI3P2MCADPDJL6JKKUSQODAYCD263AM22E2WMRIFX4A, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223512381 (6h16m25.596091736s) [noise=true , address=45.91.116.6:4973 , piece_id=HWJBA2OCLJKSN64HSRK7RULJHIQPVKIT3GWPK64Y32VUUYFEUTMQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223703747 (6h16m25.596283102s) [noise=true , address=77.90.31.17:4772 , piece_id=VP3JS4QYDXJ6R35MFRUK2XULQLGJJ23BSMM7IRWMVR3IISVEBLUQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.223956139 (6h16m25.596535495s) [noise=true , address=77.90.31.12:4592 , piece_id=36WYINRSYTUIPPO76JUD7HAO6SX4NW25G37XPOU3YVT2QLACABIA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224198243 (6h16m25.596777598s) [noise=true , address=104.36.82.5:4837 , piece_id=7YZBYKPWKCGARRJLE7WK3YRR536QC7BZZFUHYGCGQRTHRX3PTSPQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224392975 (6h16m25.59697233s) [noise=true , address=77.90.31.21:4878 , piece_id=PSLYCIRGFJ6RTL65HOEO3S2JDC737V5MJ6R3PGJJ3ODMDPNIHWCA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224586385 (6h16m25.59716574s) [noise=true , address=77.90.31.18:4807 , piece_id=3S6MGZJ2IC7UVNLPWKJCPBCY23B2DLGIJ6ANTNPNB4UKDIW45Y7Q, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.224772702 (6h16m25.597352057s) [noise=true , address=45.91.116.13:4154 , piece_id=JFXGESP7N4C4ZQ7TWKTOQVVKQ3APV6Y3C6NBN7OIKXYJMJ5RPHWA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225006720 (6h16m25.597586075s) [noise=true , address=104.36.82.7:4909 , piece_id=YCN3RDTWQZLZNYCVOQ334JY2XZJFAXDXZ5CZBWOTN4I2XI5U3FMA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225222964 (6h16m25.597802319s) [noise=true , address=45.91.116.9:4261 , piece_id=VGBFN7ZKPHS5S4SDWUHEKY6F7LS6JTVRY3ITUAGXHYWNWQMLJOIA, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:19.225416364 (6h16m25.597995719s) [noise=true , address=45.91.116.4:4939 , piece_id=BLQFFPUNAGDF7FCF2JPLBPPUAEWI7FMG4DKD7TWRMPQTYX3LEUAQ, seg_pos={0 0}, upload=4987]: Uploading piece... Jul 19 05:07:20.980351225 (6h16m27.35293058s) [upload=4987]: Next returned nil segment Jul 19 05:07:20.980380271 (6h16m27.352959626s) [upload=4987]: Waiting for error group managing segments... Jul 19 05:07:21.267793974 (6h16m27.640373329s) [noise=true , address=104.36.82.4:4801 , piece_id=3O2Q25TEDWVGN5PKMUMVK2TGYGXKRDWPV3ZY32JL7HQCDO3FB6KA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.285583227 (6h16m27.658162583s) [noise=true , address=45.91.116.13:4154 , piece_id=JFXGESP7N4C4ZQ7TWKTOQVVKQ3APV6Y3C6NBN7OIKXYJMJ5RPHWA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.286980110 (6h16m27.659559465s) [noise=true , address=45.91.116.9:4261 , piece_id=VGBFN7ZKPHS5S4SDWUHEKY6F7LS6JTVRY3ITUAGXHYWNWQMLJOIA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.359982162 (6h16m27.732561517s) [noise=true , address=104.36.82.5:4837 , piece_id=7YZBYKPWKCGARRJLE7WK3YRR536QC7BZZFUHYGCGQRTHRX3PTSPQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.363819385 (6h16m27.73639874s) [noise=true , address=45.91.116.6:4974 , piece_id=U3Z2FOR3EZI3P2MCADPDJL6JKKUSQODAYCD263AM22E2WMRIFX4A, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.372194202 (6h16m27.744773557s) [noise=true , address=45.91.116.6:4973 , piece_id=HWJBA2OCLJKSN64HSRK7RULJHIQPVKIT3GWPK64Y32VUUYFEUTMQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.375296359 (6h16m27.747875714s) [noise=true , address=104.36.82.7:4909 , piece_id=YCN3RDTWQZLZNYCVOQ334JY2XZJFAXDXZ5CZBWOTN4I2XI5U3FMA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:21.381291291 (6h16m27.753870646s) [noise=true , address=45.91.116.4:4939 , piece_id=BLQFFPUNAGDF7FCF2JPLBPPUAEWI7FMG4DKD7TWRMPQTYX3LEUAQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:32.446172843 (6h16m38.818752198s) [noise=true , address=77.90.31.17:4772 , piece_id=VP3JS4QYDXJ6R35MFRUK2XULQLGJJ23BSMM7IRWMVR3IISVEBLUQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:37.950304448 (6h16m44.322883803s) [noise=true , address=77.90.31.10:4520 , piece_id=FOH2YFXGUQ772SNJXDP7ZR7CF5LFZ3LGYKCIEOEM4LJHPSMRMYHQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:38.051458593 (6h16m44.424037949s) [noise=true , address=77.90.31.15:4697 , piece_id=544MNCQQNU6A75WX2WIJQ6A3F4N4GCCSSX3C4UTT7WHH77QOCUHA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:38.236231036 (6h16m44.608810401s) [noise=true , address=77.90.31.14:4663 , piece_id=6XQO4MRE2VYQHQAQR3BGOLYFPFDR3GNOQNO4NHNZJCQKGWVO6JAQ, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:48.143474611 (6h16m54.516053967s) [noise=true , address=77.90.31.21:4878 , piece_id=PSLYCIRGFJ6RTL65HOEO3S2JDC737V5MJ6R3PGJJ3ODMDPNIHWCA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: <nil> Jul 19 05:07:48.143499839 (6h16m54.516079194s) [seg_pos={0 0}, upload=4987]: Segment reached optimal threshold of 13 pieces. Jul 19 05:07:48.143654295 (6h16m54.51623364s) [noise=true , address=77.90.31.18:4807 , piece_id=3S6MGZJ2IC7UVNLPWKJCPBCY23B2DLGIJ6ANTNPNB4UKDIW45Y7Q, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Jul 19 05:07:48.143694843 (6h16m54.516274198s) [noise=true , address=77.90.31.12:4592 , piece_id=36WYINRSYTUIPPO76JUD7HAO6SX4NW25G37XPOU3YVT2QLACABIA, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Jul 19 05:07:48.143734959 (6h16m54.516314314s) [noise=true , address=77.90.31.15:4699 , piece_id=AGUDMA4LGNU3DSZB42QLS7M4JEZ6N5O7PNOAUOMLIQJEGISI3W5Q, seg_pos={0 0}, upload=4987]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57034->77.90.31.15:4699: use of closed network connection Jul 19 05:07:48.143753064 (6h16m54.516332419s) [seg_pos={0 0}, upload=4987]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 05:07:48.143813039 (6h16m54.516392384s) [upload=4987]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:07:48.143816736 (6h16m54.516396091s) [upload=4987]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:07:48.532152020 (6h16m54.904731375s) [upload=4987]: Done uploading segments... Jul 19 05:07:48.532172109 (6h16m54.904751464s) [upload=4987]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.226217619s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 13.318795052s FAIL (2025-07-19 05:07:23 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 13.318817465s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 24.940191671s FAIL (2025-07-19 05:07:09 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 24.940215587s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................................XX...X...X....X..X....XXXXX.] duration: 5.956742044s FAIL (2025-07-19 05:06:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ection; write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection Jul 19 05:05:41.847565883 (6h14m48.220145238s) [seg_pos={0 0}, upload=4958]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:05:42.084038342 (6h14m48.456617697s) [upload=4958]: Done uploading segments... Jul 19 05:05:42.084064663 (6h14m48.456644018s) [upload=4958]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:06:10.656276029 (6h15m17.028855385s) [upload=4971]: Starting upload Jul 19 05:06:10.656326216 (6h15m17.028905571s) [upload=4971]: Done starting upload Jul 19 05:06:10.656517231 (6h15m17.029096576s) [upload=4971]: Uploading segments... Jul 19 05:06:10.656546377 (6h15m17.029125732s) [upload=4971]: Got next segment. Inline: false Jul 19 05:06:10.656572427 (6h15m17.029151782s) [upload=4971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:06:10.656576695 (6h15m17.02915604s) [upload=4971]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:06:10.894140643 (6h15m17.266719998s) [seg_pos={0 0}, upload=4971]: Begin upload segment... Jul 19 05:06:10.894202762 (6h15m17.266782118s) [seg_pos={0 0}, upload=4971]: Done begin upload segment. Jul 19 05:06:10.894225636 (6h15m17.266804991s) [noise=true , address=45.91.116.11:4119 , piece_id=WOSNNCJJCFOVHU5LPDJJJVRFZL4JI5GGXR6PDE5I6FMWTYOHXMNQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.894239603 (6h15m17.266818968s) [noise=true , address=77.90.31.13:4627 , piece_id=W7OKPZLWR4ETITJJJQYEC5MATEVYKTOESKIW3T45AR7X7OW2PTIQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.894613498 (6h15m17.267192853s) [noise=true , address=45.91.116.12:4225 , piece_id=FEAORHV5P6FOTKFQTL57LFNJBOEXKKW6YXBV4EGPY7ENYICF4HZQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895028152 (6h15m17.267607507s) [noise=true , address=77.90.31.10:4520 , piece_id=U7J4BEWUBZXQGQ4VVGVUE7ZL4YXU5CWQF4NTDQKQBW3R2KT3ICBQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895080372 (6h15m17.267659727s) [noise=true , address=77.90.31.10:4519 , piece_id=JXXKI2ULFDUJYSL33MAUEI73QCM5F3L6LUCKHJYQGV6P2LWENLJA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895305061 (6h15m17.267884416s) [noise=true , address=45.91.116.12:4226 , piece_id=2VYSGVPZDHXFGRZJITNDPCPYU4C35VYVWWJU3SE3VZ5P7OEJE46Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895411916 (6h15m17.267991271s) [noise=true , address=45.91.116.8:4047 , piece_id=DTD2PZR72MQBK72KJLJSNHPH3I4WO3EOSTC2HMDSHZYZ66PAP6CA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895576090 (6h15m17.268155445s) [noise=true , address=77.90.31.12:4590 , piece_id=CQVAKS3RXC6JCDRKA4R3AKKBRVB4KOVW4MCRGGTAE473WQ7PN4PQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895721348 (6h15m17.268300703s) [noise=true , address=45.91.116.4:4938 , piece_id=UJKB2LQPMR2ZCABG6R5KVVRW7HXZH2K7R4C3Z4TXH73FJZDHSRHA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896095063 (6h15m17.268674418s) [noise=true , address=77.90.31.16:4734 , piece_id=4P24MNXH3SWO6AW6YZQSOAJV3MPGXO7AWV4USNZFQPVNYKLMJITQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896392553 (6h15m17.268971908s) [noise=true , address=45.91.116.9:4261 , piece_id=PCFZ7DBBS4645WSUFVRCMZREYNVWTTXP7LAFZ62M24VOUTE5NUTQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896704390 (6h15m17.269283745s) [noise=true , address=45.91.116.10:4083 , piece_id=OS4B3WPS6634RESFG5NMPK3YXBOT4FVTFFMPTYA2JOP73RCXIB2Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896735549 (6h15m17.269314904s) [noise=true , address=77.90.31.17:4771 , piece_id=GZEKXMTRPNFFMEUSED3JPONTD7WUJRQLGKEOK526PUNSBGBEOIFA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897025003 (6h15m17.269604358s) [noise=true , address=45.91.116.13:4155 , piece_id=WPYVSGXQML5FDZFMQIH53BAMGGHUSQCT3QBB622AGC6LUBRDBLLA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897329496 (6h15m17.269908851s) [noise=true , address=77.90.31.18:4807 , piece_id=VI2M6F2RX2ZDRB3VSYTRZVIP72MRANUQFJVEVJPHG6MOOEKB3JTQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897613429 (6h15m17.270192774s) [noise=true , address=45.91.116.5:4299 , piece_id=ZRFG3A3QIUKD2GDGMTJC5WOHO7WH3XEGI3OHMTN33UCMJH75BN3Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:12.629052655 (6h15m19.001632s) [upload=4971]: Next returned nil segment Jul 19 05:06:12.629069227 (6h15m19.001648583s) [upload=4971]: Waiting for error group managing segments... Jul 19 05:06:12.906950458 (6h15m19.279529813s) [noise=true , address=45.91.116.9:4261 , piece_id=PCFZ7DBBS4645WSUFVRCMZREYNVWTTXP7LAFZ62M24VOUTE5NUTQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.920268145 (6h15m19.29284749s) [noise=true , address=45.91.116.8:4047 , piece_id=DTD2PZR72MQBK72KJLJSNHPH3I4WO3EOSTC2HMDSHZYZ66PAP6CA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.925685361 (6h15m19.298264716s) [noise=true , address=45.91.116.11:4119 , piece_id=WOSNNCJJCFOVHU5LPDJJJVRFZL4JI5GGXR6PDE5I6FMWTYOHXMNQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.989285430 (6h15m19.361864785s) [noise=true , address=45.91.116.13:4155 , piece_id=WPYVSGXQML5FDZFMQIH53BAMGGHUSQCT3QBB622AGC6LUBRDBLLA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.991576755 (6h15m19.36415611s) [noise=true , address=45.91.116.12:4226 , piece_id=2VYSGVPZDHXFGRZJITNDPCPYU4C35VYVWWJU3SE3VZ5P7OEJE46Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.994417591 (6h15m19.366996946s) [noise=true , address=45.91.116.4:4938 , piece_id=UJKB2LQPMR2ZCABG6R5KVVRW7HXZH2K7R4C3Z4TXH73FJZDHSRHA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.016990598 (6h15m19.389569953s) [noise=true , address=45.91.116.5:4299 , piece_id=ZRFG3A3QIUKD2GDGMTJC5WOHO7WH3XEGI3OHMTN33UCMJH75BN3Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.061490276 (6h15m19.434069621s) [noise=true , address=45.91.116.10:4083 , piece_id=OS4B3WPS6634RESFG5NMPK3YXBOT4FVTFFMPTYA2JOP73RCXIB2Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.087738056 (6h15m19.460317411s) [noise=true , address=45.91.116.12:4225 , piece_id=FEAORHV5P6FOTKFQTL57LFNJBOEXKKW6YXBV4EGPY7ENYICF4HZQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:20.062962657 (6h15m26.435542002s) [noise=true , address=77.90.31.10:4520 , piece_id=U7J4BEWUBZXQGQ4VVGVUE7ZL4YXU5CWQF4NTDQKQBW3R2KT3ICBQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:25.282503273 (6h15m31.655082628s) [noise=true , address=77.90.31.18:4807 , piece_id=VI2M6F2RX2ZDRB3VSYTRZVIP72MRANUQFJVEVJPHG6MOOEKB3JTQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:33.431431980 (6h15m39.804011335s) [noise=true , address=77.90.31.13:4627 , piece_id=W7OKPZLWR4ETITJJJQYEC5MATEVYKTOESKIW3T45AR7X7OW2PTIQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:40.007921706 (6h15m46.380501061s) [noise=true , address=77.90.31.16:4734 , piece_id=4P24MNXH3SWO6AW6YZQSOAJV3MPGXO7AWV4USNZFQPVNYKLMJITQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:06:40.008083245 (6h15m46.380662591s) [noise=true , address=77.90.31.10:4519 , piece_id=JXXKI2ULFDUJYSL33MAUEI73QCM5F3L6LUCKHJYQGV6P2LWENLJA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection Jul 19 05:06:40.008103685 (6h15m46.38068305s) [noise=true , address=77.90.31.17:4771 , piece_id=GZEKXMTRPNFFMEUSED3JPONTD7WUJRQLGKEOK526PUNSBGBEOIFA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection Jul 19 05:06:40.008154202 (6h15m46.380733567s) [noise=true , address=77.90.31.12:4590 , piece_id=CQVAKS3RXC6JCDRKA4R3AKKBRVB4KOVW4MCRGGTAE473WQ7PN4PQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection Jul 19 05:06:40.008196201 (6h15m46.380775557s) [seg_pos={0 0}, upload=4971]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:06:40.248804847 (6h15m46.621384202s) [upload=4971]: Done uploading segments... Jul 19 05:06:40.248830677 (6h15m46.621410032s) [upload=4971]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.956762282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................XX...X...X....X..X....XXXXX] duration: 30.24274787s FAIL (2025-07-19 05:06:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ection; write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection Jul 19 05:05:41.847565883 (6h14m48.220145238s) [seg_pos={0 0}, upload=4958]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:05:42.084038342 (6h14m48.456617697s) [upload=4958]: Done uploading segments... Jul 19 05:05:42.084064663 (6h14m48.456644018s) [upload=4958]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:06:10.656276029 (6h15m17.028855385s) [upload=4971]: Starting upload Jul 19 05:06:10.656326216 (6h15m17.028905571s) [upload=4971]: Done starting upload Jul 19 05:06:10.656517231 (6h15m17.029096576s) [upload=4971]: Uploading segments... Jul 19 05:06:10.656546377 (6h15m17.029125732s) [upload=4971]: Got next segment. Inline: false Jul 19 05:06:10.656572427 (6h15m17.029151782s) [upload=4971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:06:10.656576695 (6h15m17.02915604s) [upload=4971]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:06:10.894140643 (6h15m17.266719998s) [seg_pos={0 0}, upload=4971]: Begin upload segment... Jul 19 05:06:10.894202762 (6h15m17.266782118s) [seg_pos={0 0}, upload=4971]: Done begin upload segment. Jul 19 05:06:10.894225636 (6h15m17.266804991s) [noise=true , address=45.91.116.11:4119 , piece_id=WOSNNCJJCFOVHU5LPDJJJVRFZL4JI5GGXR6PDE5I6FMWTYOHXMNQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.894239603 (6h15m17.266818968s) [noise=true , address=77.90.31.13:4627 , piece_id=W7OKPZLWR4ETITJJJQYEC5MATEVYKTOESKIW3T45AR7X7OW2PTIQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.894613498 (6h15m17.267192853s) [noise=true , address=45.91.116.12:4225 , piece_id=FEAORHV5P6FOTKFQTL57LFNJBOEXKKW6YXBV4EGPY7ENYICF4HZQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895028152 (6h15m17.267607507s) [noise=true , address=77.90.31.10:4520 , piece_id=U7J4BEWUBZXQGQ4VVGVUE7ZL4YXU5CWQF4NTDQKQBW3R2KT3ICBQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895080372 (6h15m17.267659727s) [noise=true , address=77.90.31.10:4519 , piece_id=JXXKI2ULFDUJYSL33MAUEI73QCM5F3L6LUCKHJYQGV6P2LWENLJA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895305061 (6h15m17.267884416s) [noise=true , address=45.91.116.12:4226 , piece_id=2VYSGVPZDHXFGRZJITNDPCPYU4C35VYVWWJU3SE3VZ5P7OEJE46Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895411916 (6h15m17.267991271s) [noise=true , address=45.91.116.8:4047 , piece_id=DTD2PZR72MQBK72KJLJSNHPH3I4WO3EOSTC2HMDSHZYZ66PAP6CA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895576090 (6h15m17.268155445s) [noise=true , address=77.90.31.12:4590 , piece_id=CQVAKS3RXC6JCDRKA4R3AKKBRVB4KOVW4MCRGGTAE473WQ7PN4PQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.895721348 (6h15m17.268300703s) [noise=true , address=45.91.116.4:4938 , piece_id=UJKB2LQPMR2ZCABG6R5KVVRW7HXZH2K7R4C3Z4TXH73FJZDHSRHA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896095063 (6h15m17.268674418s) [noise=true , address=77.90.31.16:4734 , piece_id=4P24MNXH3SWO6AW6YZQSOAJV3MPGXO7AWV4USNZFQPVNYKLMJITQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896392553 (6h15m17.268971908s) [noise=true , address=45.91.116.9:4261 , piece_id=PCFZ7DBBS4645WSUFVRCMZREYNVWTTXP7LAFZ62M24VOUTE5NUTQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896704390 (6h15m17.269283745s) [noise=true , address=45.91.116.10:4083 , piece_id=OS4B3WPS6634RESFG5NMPK3YXBOT4FVTFFMPTYA2JOP73RCXIB2Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.896735549 (6h15m17.269314904s) [noise=true , address=77.90.31.17:4771 , piece_id=GZEKXMTRPNFFMEUSED3JPONTD7WUJRQLGKEOK526PUNSBGBEOIFA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897025003 (6h15m17.269604358s) [noise=true , address=45.91.116.13:4155 , piece_id=WPYVSGXQML5FDZFMQIH53BAMGGHUSQCT3QBB622AGC6LUBRDBLLA, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897329496 (6h15m17.269908851s) [noise=true , address=77.90.31.18:4807 , piece_id=VI2M6F2RX2ZDRB3VSYTRZVIP72MRANUQFJVEVJPHG6MOOEKB3JTQ, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:10.897613429 (6h15m17.270192774s) [noise=true , address=45.91.116.5:4299 , piece_id=ZRFG3A3QIUKD2GDGMTJC5WOHO7WH3XEGI3OHMTN33UCMJH75BN3Q, seg_pos={0 0}, upload=4971]: Uploading piece... Jul 19 05:06:12.629052655 (6h15m19.001632s) [upload=4971]: Next returned nil segment Jul 19 05:06:12.629069227 (6h15m19.001648583s) [upload=4971]: Waiting for error group managing segments... Jul 19 05:06:12.906950458 (6h15m19.279529813s) [noise=true , address=45.91.116.9:4261 , piece_id=PCFZ7DBBS4645WSUFVRCMZREYNVWTTXP7LAFZ62M24VOUTE5NUTQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.920268145 (6h15m19.29284749s) [noise=true , address=45.91.116.8:4047 , piece_id=DTD2PZR72MQBK72KJLJSNHPH3I4WO3EOSTC2HMDSHZYZ66PAP6CA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.925685361 (6h15m19.298264716s) [noise=true , address=45.91.116.11:4119 , piece_id=WOSNNCJJCFOVHU5LPDJJJVRFZL4JI5GGXR6PDE5I6FMWTYOHXMNQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.989285430 (6h15m19.361864785s) [noise=true , address=45.91.116.13:4155 , piece_id=WPYVSGXQML5FDZFMQIH53BAMGGHUSQCT3QBB622AGC6LUBRDBLLA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.991576755 (6h15m19.36415611s) [noise=true , address=45.91.116.12:4226 , piece_id=2VYSGVPZDHXFGRZJITNDPCPYU4C35VYVWWJU3SE3VZ5P7OEJE46Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:12.994417591 (6h15m19.366996946s) [noise=true , address=45.91.116.4:4938 , piece_id=UJKB2LQPMR2ZCABG6R5KVVRW7HXZH2K7R4C3Z4TXH73FJZDHSRHA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.016990598 (6h15m19.389569953s) [noise=true , address=45.91.116.5:4299 , piece_id=ZRFG3A3QIUKD2GDGMTJC5WOHO7WH3XEGI3OHMTN33UCMJH75BN3Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.061490276 (6h15m19.434069621s) [noise=true , address=45.91.116.10:4083 , piece_id=OS4B3WPS6634RESFG5NMPK3YXBOT4FVTFFMPTYA2JOP73RCXIB2Q, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:13.087738056 (6h15m19.460317411s) [noise=true , address=45.91.116.12:4225 , piece_id=FEAORHV5P6FOTKFQTL57LFNJBOEXKKW6YXBV4EGPY7ENYICF4HZQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:20.062962657 (6h15m26.435542002s) [noise=true , address=77.90.31.10:4520 , piece_id=U7J4BEWUBZXQGQ4VVGVUE7ZL4YXU5CWQF4NTDQKQBW3R2KT3ICBQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:25.282503273 (6h15m31.655082628s) [noise=true , address=77.90.31.18:4807 , piece_id=VI2M6F2RX2ZDRB3VSYTRZVIP72MRANUQFJVEVJPHG6MOOEKB3JTQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:33.431431980 (6h15m39.804011335s) [noise=true , address=77.90.31.13:4627 , piece_id=W7OKPZLWR4ETITJJJQYEC5MATEVYKTOESKIW3T45AR7X7OW2PTIQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: <nil> Jul 19 05:06:40.007921706 (6h15m46.380501061s) [noise=true , address=77.90.31.16:4734 , piece_id=4P24MNXH3SWO6AW6YZQSOAJV3MPGXO7AWV4USNZFQPVNYKLMJITQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:06:40.008083245 (6h15m46.380662591s) [noise=true , address=77.90.31.10:4519 , piece_id=JXXKI2ULFDUJYSL33MAUEI73QCM5F3L6LUCKHJYQGV6P2LWENLJA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33330->77.90.31.10:4519: use of closed network connection Jul 19 05:06:40.008103685 (6h15m46.38068305s) [noise=true , address=77.90.31.17:4771 , piece_id=GZEKXMTRPNFFMEUSED3JPONTD7WUJRQLGKEOK526PUNSBGBEOIFA, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60096->77.90.31.17:4771: use of closed network connection Jul 19 05:06:40.008154202 (6h15m46.380733567s) [noise=true , address=77.90.31.12:4590 , piece_id=CQVAKS3RXC6JCDRKA4R3AKKBRVB4KOVW4MCRGGTAE473WQ7PN4PQ, seg_pos={0 0}, upload=4971]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33586->77.90.31.12:4590: use of closed network connection Jul 19 05:06:40.008196201 (6h15m46.380775557s) [seg_pos={0 0}, upload=4971]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:06:40.248804847 (6h15m46.621384202s) [upload=4971]: Done uploading segments... Jul 19 05:06:40.248830677 (6h15m46.621410032s) [upload=4971]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.242776065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.819614288s FAIL (2025-07-19 05:05:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.819637191s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................XX...X...X....X..X....XXXX] duration: 30.23871209s FAIL (2025-07-19 05:05:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== work connection; write tcp 188.166.225.170:38324->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38324->77.90.31.16:4735: use of closed network connection Jul 19 05:04:54.419314875 (6h14m0.79189423s) [seg_pos={0 0}, upload=4947]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:04:54.664118007 (6h14m1.036697362s) [upload=4947]: Done uploading segments... Jul 19 05:04:54.664146190 (6h14m1.036725545s) [upload=4947]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:05:12.495087221 (6h14m18.867666576s) [upload=4958]: Starting upload Jul 19 05:05:12.495145612 (6h14m18.867724967s) [upload=4958]: Done starting upload Jul 19 05:05:12.495231146 (6h14m18.867810501s) [upload=4958]: Uploading segments... Jul 19 05:05:12.495387355 (6h14m18.86796671s) [upload=4958]: Got next segment. Inline: false Jul 19 05:05:12.495419056 (6h14m18.867998401s) [upload=4958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:05:12.495424125 (6h14m18.86800347s) [upload=4958]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:05:12.816820470 (6h14m19.189399855s) [seg_pos={0 0}, upload=4958]: Begin upload segment... Jul 19 05:05:12.816882909 (6h14m19.189462264s) [seg_pos={0 0}, upload=4958]: Done begin upload segment. Jul 19 05:05:12.816899441 (6h14m19.189478796s) [noise=true , address=77.90.31.10:4517 , piece_id=NQO2OFIRACOHPDU64QETHREZHIKKXG2GPDTJE65EZ2G7IXRFPVKA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.816916042 (6h14m19.189495397s) [noise=true , address=45.91.116.4:4938 , piece_id=TVZQT5XBSRUGE4IFCCWHQY7OTC6C4X5B4FWQREYRW3KTPJE5AHRA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.816980035 (6h14m19.1895594s) [noise=true , address=45.91.116.5:4298 , piece_id=X4WMJMAB5BJBUIHHLQBMTVFZASETVMEJK6JJY23S7J4YCVTGXL2Q, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.817717386 (6h14m19.190296781s) [noise=true , address=77.90.31.17:4772 , piece_id=UE2C3F4C2J472KPRRAFL7X4X3I5RPPGFQMOXK7KDLGHFXTRXNURA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.817733768 (6h14m19.190313133s) [noise=true , address=45.91.116.10:4082 , piece_id=KFUYHCDUW5MXDXTQ2YECKHK3KJDIKUHVT5WDYJRWTVIHLAKIBSKA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.818428637 (6h14m19.191007992s) [noise=true , address=45.91.116.13:4155 , piece_id=Y7FEYDKBY56OCM6OHQOD56ZFOOHIJTRDOCYRRJ3DHIR3ZX7TLZUQ, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.819173994 (6h14m19.191753349s) [noise=true , address=77.90.31.19:4842 , piece_id=BM5SEBDLA56IZ2IVJH4RGH5CF7S5FUTMK53Z7YCU6FRCWJMRODMA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.819612493 (6h14m19.192191858s) [noise=true , address=45.91.116.14:4189 , piece_id=6Z7G5HY6Y53JTOHNCOEPF3OHXXBMYD3L46W3G3JCFTPTFF7SLWEQ, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.820070189 (6h14m19.192649544s) [noise=true , address=45.91.116.11:4117 , piece_id=ZTM4KFP5UWB3VVHPR2PBF3NS335XKCDJ2Y2C4TCQZOGLLB54MTAQ, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.820451068 (6h14m19.193030423s) [noise=true , address=77.90.31.13:4626 , piece_id=G7I2XLZMHI2XHXUE64TIOLFG2VFAIAD62YSY5KLMO57XNRIWMKIA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.820782132 (6h14m19.193361487s) [noise=true , address=45.91.116.9:4261 , piece_id=YWUFIKX2RHKZEEH2GDW5JMD2Y5FJAEETOIOKGGL3EH4KRAR2XLNA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.821090962 (6h14m19.193670317s) [noise=true , address=45.91.116.9:4263 , piece_id=RQ7AIZL22DZOGOEOTZ4S4V656SLPG3K5TKFNH7G4CBVF7DBHRL2Q, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.821410845 (6h14m19.1939902s) [noise=true , address=45.91.116.10:4081 , piece_id=JGYXIT6LBGYL5EANR4UBY4XRDI43LQNYV6PE2XQUDZ422BLLSYKQ, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.821690249 (6h14m19.194269604s) [noise=true , address=45.91.116.14:4191 , piece_id=KOC5G2RUF4ACV2ZB2BWHTT3IJ43ILY4ZJ7NIO3CGUI6V6NAZ6QGA, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.821990504 (6h14m19.194569859s) [noise=true , address=77.90.31.19:4841 , piece_id=W76GRVUXSP267PDUM257GJMOCATWFKXC2YYTXQQT6HDDIMNYPWVQ, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:12.823132950 (6h14m19.195712315s) [noise=true , address=77.90.31.10:4520 , piece_id=CVS347QAQH36X3UZTCFQZAGXHA4PMGJRWHGPXQXDZKJIBQLUWG3A, seg_pos={0 0}, upload=4958]: Uploading piece... Jul 19 05:05:14.557564334 (6h14m20.930143689s) [upload=4958]: Next returned nil segment Jul 19 05:05:14.557585223 (6h14m20.930164579s) [upload=4958]: Waiting for error group managing segments... Jul 19 05:05:14.833205730 (6h14m21.205785085s) [noise=true , address=45.91.116.10:4081 , piece_id=JGYXIT6LBGYL5EANR4UBY4XRDI43LQNYV6PE2XQUDZ422BLLSYKQ, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.836674397 (6h14m21.209253753s) [noise=true , address=45.91.116.5:4298 , piece_id=X4WMJMAB5BJBUIHHLQBMTVFZASETVMEJK6JJY23S7J4YCVTGXL2Q, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.863812671 (6h14m21.236392026s) [noise=true , address=45.91.116.4:4938 , piece_id=TVZQT5XBSRUGE4IFCCWHQY7OTC6C4X5B4FWQREYRW3KTPJE5AHRA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.921764211 (6h14m21.294343566s) [noise=true , address=45.91.116.9:4261 , piece_id=YWUFIKX2RHKZEEH2GDW5JMD2Y5FJAEETOIOKGGL3EH4KRAR2XLNA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.926647236 (6h14m21.299226591s) [noise=true , address=45.91.116.10:4082 , piece_id=KFUYHCDUW5MXDXTQ2YECKHK3KJDIKUHVT5WDYJRWTVIHLAKIBSKA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.932926532 (6h14m21.305505887s) [noise=true , address=45.91.116.13:4155 , piece_id=Y7FEYDKBY56OCM6OHQOD56ZFOOHIJTRDOCYRRJ3DHIR3ZX7TLZUQ, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:14.951446273 (6h14m21.324025628s) [noise=true , address=45.91.116.14:4191 , piece_id=KOC5G2RUF4ACV2ZB2BWHTT3IJ43ILY4ZJ7NIO3CGUI6V6NAZ6QGA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:15.009071949 (6h14m21.381651304s) [noise=true , address=45.91.116.14:4189 , piece_id=6Z7G5HY6Y53JTOHNCOEPF3OHXXBMYD3L46W3G3JCFTPTFF7SLWEQ, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:15.009885836 (6h14m21.382465191s) [noise=true , address=45.91.116.9:4263 , piece_id=RQ7AIZL22DZOGOEOTZ4S4V656SLPG3K5TKFNH7G4CBVF7DBHRL2Q, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:15.026495553 (6h14m21.399074908s) [noise=true , address=45.91.116.11:4117 , piece_id=ZTM4KFP5UWB3VVHPR2PBF3NS335XKCDJ2Y2C4TCQZOGLLB54MTAQ, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:27.517038725 (6h14m33.88961808s) [noise=true , address=77.90.31.19:4842 , piece_id=BM5SEBDLA56IZ2IVJH4RGH5CF7S5FUTMK53Z7YCU6FRCWJMRODMA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:34.774342491 (6h14m41.146921846s) [noise=true , address=77.90.31.13:4626 , piece_id=G7I2XLZMHI2XHXUE64TIOLFG2VFAIAD62YSY5KLMO57XNRIWMKIA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: <nil> Jul 19 05:05:41.847054935 (6h14m48.21963429s) [noise=true , address=77.90.31.19:4841 , piece_id=W76GRVUXSP267PDUM257GJMOCATWFKXC2YYTXQQT6HDDIMNYPWVQ, seg_pos={0 0}, upload=4958]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:54862->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:54862->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54862->77.90.31.19:4841: use of closed network connection Jul 19 05:05:41.847278974 (6h14m48.219858329s) [noise=true , address=77.90.31.17:4772 , piece_id=UE2C3F4C2J472KPRRAFL7X4X3I5RPPGFQMOXK7KDLGHFXTRXNURA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:05:41.847473375 (6h14m48.22005274s) [noise=true , address=77.90.31.10:4520 , piece_id=CVS347QAQH36X3UZTCFQZAGXHA4PMGJRWHGPXQXDZKJIBQLUWG3A, seg_pos={0 0}, upload=4958]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 188.166.225.170:52278->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:52278->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52278->77.90.31.10:4520: use of closed network connection Jul 19 05:05:41.847546345 (6h14m48.2201257s) [noise=true , address=77.90.31.10:4517 , piece_id=NQO2OFIRACOHPDU64QETHREZHIKKXG2GPDTJE65EZ2G7IXRFPVKA, seg_pos={0 0}, upload=4958]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36188->77.90.31.10:4517: use of closed network connection Jul 19 05:05:41.847565883 (6h14m48.220145238s) [seg_pos={0 0}, upload=4958]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:05:42.084038342 (6h14m48.456617697s) [upload=4958]: Done uploading segments... Jul 19 05:05:42.084064663 (6h14m48.456644018s) [upload=4958]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.238737107s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................XX...X...X....X..X....XXX] duration: 30.246783778s FAIL (2025-07-19 05:04:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 01493142 (6h13m20.574072497s) [seg_pos={0 0}, upload=4939]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:04:14.442040800 (6h13m20.814620155s) [upload=4939]: Done uploading segments... Jul 19 05:04:14.442067992 (6h13m20.814647347s) [upload=4939]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:04:25.062975524 (6h13m31.435554889s) [upload=4947]: Starting upload Jul 19 05:04:25.063050326 (6h13m31.435629681s) [upload=4947]: Done starting upload Jul 19 05:04:25.063155357 (6h13m31.435734712s) [upload=4947]: Uploading segments... Jul 19 05:04:25.063237875 (6h13m31.43581723s) [upload=4947]: Got next segment. Inline: false Jul 19 05:04:25.063271609 (6h13m31.435850964s) [upload=4947]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:04:25.063276660 (6h13m31.435856015s) [upload=4947]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:04:25.294241168 (6h13m31.666820523s) [seg_pos={0 0}, upload=4947]: Begin upload segment... Jul 19 05:04:25.294284011 (6h13m31.666863366s) [seg_pos={0 0}, upload=4947]: Done begin upload segment. Jul 19 05:04:25.294297586 (6h13m31.666876941s) [noise=true , address=77.90.31.11:4553 , piece_id=TB7IAF77R54HUVN77BVIF5J4Y6GTUBYYYJD2ATFAP2L3TN7CMEHQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294308607 (6h13m31.666887962s) [noise=true , address=45.91.116.7:4009 , piece_id=MHW6WZXKPFHZSRFX5CSZCBIYFKUDJEZ46JUNXELGMNUP4Q5YXIFQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294349495 (6h13m31.66692885s) [noise=true , address=45.91.116.14:4190 , piece_id=VXT7XAAIMZ33NIUM5GCDY7CTOFRH6H2SKSUMRHWBKTKBBVQ3NWJA, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294525852 (6h13m31.667105208s) [noise=true , address=77.90.31.16:4735 , piece_id=35IMYCGD4WKCD4HHMZKV5OEF4JUDQ72QWZ5I7B7FUDLJPSKLV3AQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294652816 (6h13m31.667232171s) [noise=true , address=45.91.116.8:4048 , piece_id=B7KQRCPQUXLWO5AEM2EV5XOQJBRL2KLTS6J3NZZJWBFIHWWQQOGA, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294796310 (6h13m31.667375665s) [noise=true , address=77.90.31.19:4842 , piece_id=XNG3OIQXHGZF2OYASIXC7YWD6XRFU7RSMGNLGIIFZTXEBRD6LHGQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.294904327 (6h13m31.667483682s) [noise=true , address=104.36.82.9:4006 , piece_id=54FLFMKVSDJTLBFNVHPMLJOY3IAIKNOAQX3GQOPJXZTE3SZMRACA, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.295086065 (6h13m31.66766541s) [noise=true , address=45.91.116.5:4299 , piece_id=6PDEJKI6ZRKU46NJ4HE4UQ6TLVHVGOLVUZROPKD2XGKJEILNY2CQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.295349309 (6h13m31.667928664s) [noise=true , address=45.91.116.4:4939 , piece_id=A5KWQKRQSRGHKKHIKCIUNYSUFX4ONBB2CTDF2OUISTCIIY34NZ4A, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.295686304 (6h13m31.668265659s) [noise=true , address=77.90.31.13:4627 , piece_id=SAVOTAM3SQIMOYVP25MMBSYMHF73E245IB75GZW27NOGHNIDZY3A, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.295926694 (6h13m31.668506049s) [noise=true , address=45.91.116.13:4155 , piece_id=UYBNTGQBRDKWCV5DGKKUUSM7Q2E6YQAG4BARI6JLROWNT6GFQRCQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.296202381 (6h13m31.668781736s) [noise=true , address=45.91.116.7:4011 , piece_id=4R5YUFDQTB7CKHHBNVX5ZTQQW2NF54GS6YVBX2WB2EAMSB5GO5PQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.296398957 (6h13m31.668978312s) [noise=true , address=45.91.116.6:4974 , piece_id=KM7KSHTFV67VDGFDDYQXESN4GH3ZSPDDTEE5YOD437SQGAM26FCQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.296599071 (6h13m31.669178426s) [noise=true , address=45.91.116.5:4297 , piece_id=BPPHDW35GVW5YTYYJHVIWZYAHDT43VBIIC3PH7LSK76QBD7W5IYQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.296912350 (6h13m31.669491705s) [noise=true , address=77.90.31.11:4555 , piece_id=WMGKK3PJES55VZXDFTFCVJJSHX3ET3GCZ7UGAR4X6MALUON75DMQ, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:25.297121510 (6h13m31.669700865s) [noise=true , address=45.91.116.14:4191 , piece_id=YGSRZZRTE7VREHQKQZSHT6D5ZO7OYQKKEYRJ56K7JOOTK6U6XYUA, seg_pos={0 0}, upload=4947]: Uploading piece... Jul 19 05:04:27.019208544 (6h13m33.391787899s) [upload=4947]: Next returned nil segment Jul 19 05:04:27.019228112 (6h13m33.391807457s) [upload=4947]: Waiting for error group managing segments... Jul 19 05:04:27.295342934 (6h13m33.667922289s) [noise=true , address=45.91.116.5:4299 , piece_id=6PDEJKI6ZRKU46NJ4HE4UQ6TLVHVGOLVUZROPKD2XGKJEILNY2CQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.313593741 (6h13m33.686173096s) [noise=true , address=45.91.116.14:4190 , piece_id=VXT7XAAIMZ33NIUM5GCDY7CTOFRH6H2SKSUMRHWBKTKBBVQ3NWJA, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.319297996 (6h13m33.691877341s) [noise=true , address=45.91.116.13:4155 , piece_id=UYBNTGQBRDKWCV5DGKKUUSM7Q2E6YQAG4BARI6JLROWNT6GFQRCQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.320901415 (6h13m33.69348077s) [noise=true , address=45.91.116.7:4011 , piece_id=4R5YUFDQTB7CKHHBNVX5ZTQQW2NF54GS6YVBX2WB2EAMSB5GO5PQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.399554451 (6h13m33.772133806s) [noise=true , address=45.91.116.4:4939 , piece_id=A5KWQKRQSRGHKKHIKCIUNYSUFX4ONBB2CTDF2OUISTCIIY34NZ4A, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.406268059 (6h13m33.778847414s) [noise=true , address=45.91.116.5:4297 , piece_id=BPPHDW35GVW5YTYYJHVIWZYAHDT43VBIIC3PH7LSK76QBD7W5IYQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.406522836 (6h13m33.779102191s) [noise=true , address=45.91.116.14:4191 , piece_id=YGSRZZRTE7VREHQKQZSHT6D5ZO7OYQKKEYRJ56K7JOOTK6U6XYUA, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.413032854 (6h13m33.785612209s) [noise=true , address=45.91.116.8:4048 , piece_id=B7KQRCPQUXLWO5AEM2EV5XOQJBRL2KLTS6J3NZZJWBFIHWWQQOGA, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.455474905 (6h13m33.82805426s) [noise=true , address=104.36.82.9:4006 , piece_id=54FLFMKVSDJTLBFNVHPMLJOY3IAIKNOAQX3GQOPJXZTE3SZMRACA, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.461903236 (6h13m33.834482591s) [noise=true , address=45.91.116.6:4974 , piece_id=KM7KSHTFV67VDGFDDYQXESN4GH3ZSPDDTEE5YOD437SQGAM26FCQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:27.482739179 (6h13m33.855318534s) [noise=true , address=45.91.116.7:4009 , piece_id=MHW6WZXKPFHZSRFX5CSZCBIYFKUDJEZ46JUNXELGMNUP4Q5YXIFQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:41.625353298 (6h13m47.997932693s) [noise=true , address=77.90.31.19:4842 , piece_id=XNG3OIQXHGZF2OYASIXC7YWD6XRFU7RSMGNLGIIFZTXEBRD6LHGQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: <nil> Jul 19 05:04:54.419205786 (6h14m0.791785141s) [noise=true , address=77.90.31.11:4553 , piece_id=TB7IAF77R54HUVN77BVIF5J4Y6GTUBYYYJD2ATFAP2L3TN7CMEHQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:60420->77.90.31.11:4553: use of closed network connection; write tcp 188.166.225.170:60420->77.90.31.11:4553: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60420->77.90.31.11:4553: use of closed network connection Jul 19 05:04:54.419245552 (6h14m0.791824897s) [noise=true , address=77.90.31.11:4555 , piece_id=WMGKK3PJES55VZXDFTFCVJJSHX3ET3GCZ7UGAR4X6MALUON75DMQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:44430->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:44430->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44430->77.90.31.11:4555: use of closed network connection Jul 19 05:04:54.419257736 (6h14m0.791837091s) [noise=true , address=77.90.31.13:4627 , piece_id=SAVOTAM3SQIMOYVP25MMBSYMHF73E245IB75GZW27NOGHNIDZY3A, seg_pos={0 0}, upload=4947]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: write tcp 188.166.225.170:39584->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:39584->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39584->77.90.31.13:4627: use of closed network connection Jul 19 05:04:54.419271111 (6h14m0.791850466s) [noise=true , address=77.90.31.16:4735 , piece_id=35IMYCGD4WKCD4HHMZKV5OEF4JUDQ72QWZ5I7B7FUDLJPSKLV3AQ, seg_pos={0 0}, upload=4947]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:38324->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:38324->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38324->77.90.31.16:4735: use of closed network connection Jul 19 05:04:54.419314875 (6h14m0.79189423s) [seg_pos={0 0}, upload=4947]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:04:54.664118007 (6h14m1.036697362s) [upload=4947]: Done uploading segments... Jul 19 05:04:54.664146190 (6h14m1.036725545s) [upload=4947]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.246807092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.770678791s FAIL (2025-07-19 05:04:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.770703919s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................XX...X...X....X..X....XX] duration: 30.241646279s FAIL (2025-07-19 05:04:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 4925]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection Jul 19 05:03:06.533247682 (6h12m12.905827037s) [seg_pos={0 0}, upload=4925]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:03:06.775784086 (6h12m13.148363451s) [upload=4925]: Done uploading segments... Jul 19 05:03:06.775815406 (6h12m13.148394791s) [upload=4925]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 19 05:03:44.864120299 (6h12m51.236699654s) [upload=4939]: Starting upload Jul 19 05:03:44.864182970 (6h12m51.236762325s) [upload=4939]: Done starting upload Jul 19 05:03:44.864276959 (6h12m51.236856304s) [upload=4939]: Uploading segments... Jul 19 05:03:44.864413570 (6h12m51.236992915s) [upload=4939]: Got next segment. Inline: false Jul 19 05:03:44.864447625 (6h12m51.23702698s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:03:44.864452055 (6h12m51.23703141s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:03:45.115665474 (6h12m51.488244829s) [seg_pos={0 0}, upload=4939]: Begin upload segment... Jul 19 05:03:45.115718175 (6h12m51.48829753s) [seg_pos={0 0}, upload=4939]: Done begin upload segment. Jul 19 05:03:45.115741389 (6h12m51.488320744s) [noise=true , address=104.36.82.13:4186 , piece_id=4ZPHE5YJRIVLNUC3YF2JZQTM5EMIFAR5FR4F6X62KEWBCLCF66PQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.115795513 (6h12m51.488374868s) [noise=true , address=77.90.31.13:4628 , piece_id=FCMOWACIWBGSSFVHV22Q7V2KIAY27EHE3CKIHX5KAYTZNLEHQEYA, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.116158808 (6h12m51.488738163s) [noise=true , address=45.91.116.14:4191 , piece_id=TUJXONY7RDRBJQ32GHGYUVMHZYQNX3HGR76WVA2Q5VUQK5ELJ3FQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.116333593 (6h12m51.488912938s) [noise=true , address=45.91.116.9:4262 , piece_id=SY6ZMA7RBXTYTTUDG7CUSUIBDQDH6KSUYQEVGOF2277W3VBHNXPQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.116443223 (6h12m51.489022578s) [noise=true , address=77.90.31.16:4734 , piece_id=K3GMMB3LZEBGY7SK3Y4SYOCP53ZQDJE6A7QKHEAPHZJ5FPHS7SXA, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.116737546 (6h12m51.489316891s) [noise=true , address=77.90.31.21:4878 , piece_id=33ICURJROGZ7AAUL2V4TF7DGUBZVFMIILOZK32JJLM7H7YSNPDDQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.117597912 (6h12m51.490177267s) [noise=true , address=104.36.82.10:4078 , piece_id=ZVL3QTM2S6SAKLPT3BZJJUED7XEBBH72XBM6YCAWOP7EAF5VENWA, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.118828717 (6h12m51.491408072s) [noise=true , address=77.90.31.12:4592 , piece_id=WNVERLNK3OAHNCRZPTJBWUZ4TGUSIZGE2MTXZN2VVGKGBLOYV5FQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.119518137 (6h12m51.492097492s) [noise=true , address=45.91.116.14:4189 , piece_id=5VTVNCGJXELSPFW27IYSMX2AHBSOZF65K33VTL3CQCH3BD7XUKJA, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.119617917 (6h12m51.492197272s) [noise=true , address=45.91.116.4:4940 , piece_id=TSPQDQ7BPAXCAWR3BRPWFICR5I2AH4NH74K2SZDYSS5TFCGMPHKQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.119930426 (6h12m51.492509781s) [noise=true , address=104.36.82.3:4765 , piece_id=PMWXLX4WAM4PQOHOZNI7IBMD4AXZ6WPDHI6EYUWQA7X3RI7RDCKA, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.120276017 (6h12m51.492855372s) [noise=true , address=104.36.82.6:4875 , piece_id=SZFDTUWZDOLEDJIF2HRIKE7O2CFC5OQAPKOQTTL45O4PTI5SNA3Q, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.120553368 (6h12m51.493132723s) [noise=true , address=45.91.116.11:4117 , piece_id=R7A7PQWO3YJ4HSGJPCKYYH6OMTV3PFGSUM7QRTEMGTUQISKOYYZQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.120849865 (6h12m51.493429221s) [noise=true , address=104.36.82.7:4910 , piece_id=UBFAZYTQKRTQK23WRUE2YQCHEDXPIZNEQ622SYKGMWW7YGWL465A, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.121138487 (6h12m51.493717842s) [noise=true , address=45.91.116.5:4299 , piece_id=MTSIPD6QVY3UCK2L4ZDZ4UKMOIDMZFWQVCWFQNFLUV2LOWHP76NQ, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:45.121423393 (6h12m51.494002748s) [noise=true , address=77.90.31.17:4770 , piece_id=6IX73G3QBSK7LLIRRV25Q2SXXZTQFSJQYVAKUTNMKBI6KSOSEB5A, seg_pos={0 0}, upload=4939]: Uploading piece... Jul 19 05:03:46.836034706 (6h12m53.208614051s) [upload=4939]: Next returned nil segment Jul 19 05:03:46.836053923 (6h12m53.208633278s) [upload=4939]: Waiting for error group managing segments... Jul 19 05:03:47.166909631 (6h12m53.539488986s) [noise=true , address=104.36.82.10:4078 , piece_id=ZVL3QTM2S6SAKLPT3BZJJUED7XEBBH72XBM6YCAWOP7EAF5VENWA, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.170976693 (6h12m53.543556048s) [noise=true , address=45.91.116.14:4191 , piece_id=TUJXONY7RDRBJQ32GHGYUVMHZYQNX3HGR76WVA2Q5VUQK5ELJ3FQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.175582207 (6h12m53.548161552s) [noise=true , address=45.91.116.11:4117 , piece_id=R7A7PQWO3YJ4HSGJPCKYYH6OMTV3PFGSUM7QRTEMGTUQISKOYYZQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.181418145 (6h12m53.5539975s) [noise=true , address=45.91.116.4:4940 , piece_id=TSPQDQ7BPAXCAWR3BRPWFICR5I2AH4NH74K2SZDYSS5TFCGMPHKQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.191436166 (6h12m53.564015521s) [noise=true , address=45.91.116.14:4189 , piece_id=5VTVNCGJXELSPFW27IYSMX2AHBSOZF65K33VTL3CQCH3BD7XUKJA, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.239695198 (6h12m53.612274553s) [noise=true , address=104.36.82.13:4186 , piece_id=4ZPHE5YJRIVLNUC3YF2JZQTM5EMIFAR5FR4F6X62KEWBCLCF66PQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.247479896 (6h12m53.620059301s) [noise=true , address=45.91.116.9:4262 , piece_id=SY6ZMA7RBXTYTTUDG7CUSUIBDQDH6KSUYQEVGOF2277W3VBHNXPQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.254180889 (6h12m53.626760244s) [noise=true , address=104.36.82.7:4910 , piece_id=UBFAZYTQKRTQK23WRUE2YQCHEDXPIZNEQ622SYKGMWW7YGWL465A, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.257135352 (6h12m53.629714697s) [noise=true , address=104.36.82.6:4875 , piece_id=SZFDTUWZDOLEDJIF2HRIKE7O2CFC5OQAPKOQTTL45O4PTI5SNA3Q, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.268904665 (6h12m53.64148402s) [noise=true , address=104.36.82.3:4765 , piece_id=PMWXLX4WAM4PQOHOZNI7IBMD4AXZ6WPDHI6EYUWQA7X3RI7RDCKA, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:03:47.288612540 (6h12m53.661191885s) [noise=true , address=45.91.116.5:4299 , piece_id=MTSIPD6QVY3UCK2L4ZDZ4UKMOIDMZFWQVCWFQNFLUV2LOWHP76NQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:04:13.421838111 (6h13m19.794417466s) [noise=true , address=77.90.31.12:4592 , piece_id=WNVERLNK3OAHNCRZPTJBWUZ4TGUSIZGE2MTXZN2VVGKGBLOYV5FQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: <nil> Jul 19 05:04:14.201414481 (6h13m20.573993836s) [noise=true , address=77.90.31.21:4878 , piece_id=33ICURJROGZ7AAUL2V4TF7DGUBZVFMIILOZK32JJLM7H7YSNPDDQ, seg_pos={0 0}, upload=4939]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:48628->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:48628->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48628->77.90.31.21:4878: use of closed network connection Jul 19 05:04:14.201449188 (6h13m20.574028543s) [noise=true , address=77.90.31.16:4734 , piece_id=K3GMMB3LZEBGY7SK3Y4SYOCP53ZQDJE6A7QKHEAPHZJ5FPHS7SXA, seg_pos={0 0}, upload=4939]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:04:14.201455690 (6h13m20.574035045s) [noise=true , address=77.90.31.17:4770 , piece_id=6IX73G3QBSK7LLIRRV25Q2SXXZTQFSJQYVAKUTNMKBI6KSOSEB5A, seg_pos={0 0}, upload=4939]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 05:04:14.201467092 (6h13m20.574046447s) [noise=true , address=77.90.31.13:4628 , piece_id=FCMOWACIWBGSSFVHV22Q7V2KIAY27EHE3CKIHX5KAYTZNLEHQEYA, seg_pos={0 0}, upload=4939]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:44876->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:44876->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44876->77.90.31.13:4628: use of closed network connection Jul 19 05:04:14.201493142 (6h13m20.574072497s) [seg_pos={0 0}, upload=4939]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:04:14.442040800 (6h13m20.814620155s) [upload=4939]: Done uploading segments... Jul 19 05:04:14.442067992 (6h13m20.814647347s) [upload=4939]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.241669144s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 9.995585553s FAIL (2025-07-19 05:03:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 9.995608447s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....................................XX...X...X....X..X....X] duration: 30.245110122s FAIL (2025-07-19 05:03:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 4909]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 05:01:28.708760519 (6h10m35.081339874s) [upload=4909]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 05:01:29.082227724 (6h10m35.454807079s) [upload=4909]: Done uploading segments... Jul 19 05:01:29.082245619 (6h10m35.454824974s) [upload=4909]: Upload finished. err: <nil> Jul 19 05:02:37.175498145 (6h11m43.54807751s) [upload=4925]: Starting upload Jul 19 05:02:37.175628474 (6h11m43.548207829s) [upload=4925]: Done starting upload Jul 19 05:02:37.175717755 (6h11m43.54829713s) [upload=4925]: Uploading segments... Jul 19 05:02:37.176047947 (6h11m43.548627332s) [upload=4925]: Got next segment. Inline: false Jul 19 05:02:37.176090769 (6h11m43.548670154s) [upload=4925]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 05:02:37.176097472 (6h11m43.548676847s) [upload=4925]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 05:02:37.420416547 (6h11m43.792995902s) [seg_pos={0 0}, upload=4925]: Begin upload segment... Jul 19 05:02:37.420466282 (6h11m43.793045637s) [seg_pos={0 0}, upload=4925]: Done begin upload segment. Jul 19 05:02:37.420489857 (6h11m43.793069202s) [noise=true , address=77.90.31.21:4878 , piece_id=CER5K5SB2SHVA56LHYBTV4XGOETIL2TCYZFPKAS2JRNU466LEWGA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.420539963 (6h11m43.793119318s) [noise=true , address=77.90.31.15:4697 , piece_id=GTPV46CWTH2S7AR5AXS7NEOPKPLU7RPIRL3XCZRJD3WQNODZSPLA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.420887918 (6h11m43.793467273s) [noise=true , address=45.91.116.14:4190 , piece_id=S7GEO6VXAPJFU6M2LVY525SSDLUM2SNP73AI56A7ZGTXNHWS6RJA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.421075027 (6h11m43.793654382s) [noise=true , address=104.36.82.9:4007 , piece_id=6MNBAMZJ4VBJYVGPTN35J2JSKQBKXZX2JUMQHKNZA7OE36GUYEXA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.421166572 (6h11m43.793745927s) [noise=true , address=45.91.116.5:4298 , piece_id=TAA2NYNNGYVYNQ5BKJ6EBJ3UMI737LMV6SLAYGQ7FFCORK4CZS4Q, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.421410068 (6h11m43.793989423s) [noise=true , address=77.90.31.15:4699 , piece_id=MR3AYIL46SZWXIRAV7JVFYEWGREP4ME4BZTV3LWTVR2TA4JBUJ3Q, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.421493167 (6h11m43.794072522s) [noise=true , address=45.91.116.12:4226 , piece_id=E7XXPGDOGRSRDA3AUBZJKNEZGBXMJBFZGSM4JC4M2BTEUAKMCL5Q, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.421855060 (6h11m43.794434405s) [noise=true , address=45.91.116.11:4118 , piece_id=DWY3A3QGAB7SCX473QLO55UNZSIXYZRE7NO5EPGEX4FA3JCTRQPQ, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.422167978 (6h11m43.794747333s) [noise=true , address=45.91.116.9:4263 , piece_id=5RW6U4FPSZHUZITQKBOV3INKLXLOQX2SK2CCXKYMVAGJ7BMFHA3A, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.422434749 (6h11m43.795014104s) [noise=true , address=77.90.31.16:4735 , piece_id=PQKOY6ZW7FGMFZOKR3Q54AZD2725ZHYVMCJG6R6DPVRFXLP6T45Q, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.422642046 (6h11m43.795221401s) [noise=true , address=77.90.31.14:4662 , piece_id=F5X75EONQ5C7XJSXCF7TMILNIV4NDJKWYO52U3U2OYKBJV3MQY4A, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.422903305 (6h11m43.79548266s) [noise=true , address=77.90.31.16:4734 , piece_id=E3N2BN6ICSXU7VQABIMXCGRVNPIPBZFQYT6YQGKDVO2VD2XIG3OA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.423135039 (6h11m43.795714394s) [noise=true , address=77.90.31.14:4663 , piece_id=IXQ3GNG5TP2PDVGOBAAYOCIA7SCOEXOXHSVDJJVZVXN66D2KPZMA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.423409434 (6h11m43.795988789s) [noise=true , address=45.91.116.12:4227 , piece_id=WWGQZ6F4LQB6RFU6FL2ZFG5LQNDOC6XDXBR6REXV2EFG3GZ3KGXA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.423812125 (6h11m43.79639148s) [noise=true , address=77.90.31.10:4519 , piece_id=FYTPDXUOHWNUTDET25WYASKHUTNFKA44AUR6ZCL7DU2QAUC2TSEA, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:37.424165982 (6h11m43.796745337s) [noise=true , address=45.91.116.11:4117 , piece_id=YF4ASOSW6EX5A2N6QJCZ2JPTZH3CLKEWGRZYQSCIGOAKKVS56YRQ, seg_pos={0 0}, upload=4925]: Uploading piece... Jul 19 05:02:39.183548694 (6h11m45.556128049s) [upload=4925]: Next returned nil segment Jul 19 05:02:39.183567671 (6h11m45.556147026s) [upload=4925]: Waiting for error group managing segments... Jul 19 05:02:39.466183052 (6h11m45.838762407s) [noise=true , address=45.91.116.12:4226 , piece_id=E7XXPGDOGRSRDA3AUBZJKNEZGBXMJBFZGSM4JC4M2BTEUAKMCL5Q, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.469980459 (6h11m45.842559824s) [noise=true , address=45.91.116.14:4190 , piece_id=S7GEO6VXAPJFU6M2LVY525SSDLUM2SNP73AI56A7ZGTXNHWS6RJA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.480368338 (6h11m45.852947693s) [noise=true , address=45.91.116.5:4298 , piece_id=TAA2NYNNGYVYNQ5BKJ6EBJ3UMI737LMV6SLAYGQ7FFCORK4CZS4Q, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.489731406 (6h11m45.862310761s) [noise=true , address=104.36.82.9:4007 , piece_id=6MNBAMZJ4VBJYVGPTN35J2JSKQBKXZX2JUMQHKNZA7OE36GUYEXA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.553117096 (6h11m45.925696441s) [noise=true , address=45.91.116.11:4117 , piece_id=YF4ASOSW6EX5A2N6QJCZ2JPTZH3CLKEWGRZYQSCIGOAKKVS56YRQ, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.553961081 (6h11m45.926540436s) [noise=true , address=45.91.116.11:4118 , piece_id=DWY3A3QGAB7SCX473QLO55UNZSIXYZRE7NO5EPGEX4FA3JCTRQPQ, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.564133598 (6h11m45.936712953s) [noise=true , address=45.91.116.9:4263 , piece_id=5RW6U4FPSZHUZITQKBOV3INKLXLOQX2SK2CCXKYMVAGJ7BMFHA3A, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:39.564509658 (6h11m45.937089013s) [noise=true , address=45.91.116.12:4227 , piece_id=WWGQZ6F4LQB6RFU6FL2ZFG5LQNDOC6XDXBR6REXV2EFG3GZ3KGXA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:02:49.175614184 (6h11m55.548193549s) [noise=true , address=77.90.31.14:4662 , piece_id=F5X75EONQ5C7XJSXCF7TMILNIV4NDJKWYO52U3U2OYKBJV3MQY4A, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:03:01.490571726 (6h12m7.863151081s) [noise=true , address=77.90.31.10:4519 , piece_id=FYTPDXUOHWNUTDET25WYASKHUTNFKA44AUR6ZCL7DU2QAUC2TSEA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:03:03.150509176 (6h12m9.523088531s) [noise=true , address=77.90.31.15:4697 , piece_id=GTPV46CWTH2S7AR5AXS7NEOPKPLU7RPIRL3XCZRJD3WQNODZSPLA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:03:04.580327203 (6h12m10.952906558s) [noise=true , address=77.90.31.21:4878 , piece_id=CER5K5SB2SHVA56LHYBTV4XGOETIL2TCYZFPKAS2JRNU466LEWGA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: <nil> Jul 19 05:03:06.533087545 (6h12m12.905666901s) [noise=true , address=77.90.31.16:4735 , piece_id=PQKOY6ZW7FGMFZOKR3Q54AZD2725ZHYVMCJG6R6DPVRFXLP6T45Q, seg_pos={0 0}, upload=4925]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 188.166.225.170:46634->77.90.31.16:4735: use of closed network connection; write tcp 188.166.225.170:46634->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46634->77.90.31.16:4735: use of closed network connection Jul 19 05:03:06.533150576 (6h12m12.905729931s) [noise=true , address=77.90.31.16:4734 , piece_id=E3N2BN6ICSXU7VQABIMXCGRVNPIPBZFQYT6YQGKDVO2VD2XIG3OA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:33108->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:33108->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33108->77.90.31.16:4734: use of closed network connection Jul 19 05:03:06.533186534 (6h12m12.905765889s) [noise=true , address=77.90.31.14:4663 , piece_id=IXQ3GNG5TP2PDVGOBAAYOCIA7SCOEXOXHSVDJJVZVXN66D2KPZMA, seg_pos={0 0}, upload=4925]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:53054->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:53054->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53054->77.90.31.14:4663: use of closed network connection Jul 19 05:03:06.533211522 (6h12m12.905790877s) [noise=true , address=77.90.31.15:4699 , piece_id=MR3AYIL46SZWXIRAV7JVFYEWGREP4ME4BZTV3LWTVR2TA4JBUJ3Q, seg_pos={0 0}, upload=4925]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51946->77.90.31.15:4699: use of closed network connection Jul 19 05:03:06.533247682 (6h12m12.905827037s) [seg_pos={0 0}, upload=4925]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 05:03:06.775784086 (6h12m13.148363451s) [upload=4925]: Done uploading segments... Jul 19 05:03:06.775815406 (6h12m13.148394791s) [upload=4925]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.245134669s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.613068822s FAIL (2025-07-19 05:02:40 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.613094321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.01184914s FAIL (2025-07-19 05:01:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 6.011880099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.756132528s FAIL (2025-07-19 05:01:36 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.756145573s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................XX...X...X....X..X....] duration: 7.497508596s FAIL (2025-07-19 05:01:31 UTC) orbiter warning: ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== :1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: read tcp 188.166.225.170:50230->77.90.31.19:4841: use of closed network connection; context canceled Jul 19 04:56:54.832707046 (6h6m1.205286401s) [seg_pos={0 0}, upload=4846]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:56:54.832779073 (6h6m1.205358428s) [upload=4846]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:56:54.832783182 (6h6m1.205362527s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:56:55.133341022 (6h6m1.505920377s) [upload=4846]: Done uploading segments... Jul 19 04:56:55.133358805 (6h6m1.50593816s) [upload=4846]: Upload finished. err: <nil> Jul 19 04:57:50.449535830 (6h6m56.822115185s) [upload=4860]: Starting upload Jul 19 04:57:50.449598590 (6h6m56.822177935s) [upload=4860]: Done starting upload Jul 19 04:57:50.449668684 (6h6m56.822248039s) [upload=4860]: Uploading segments... Jul 19 04:57:50.449781009 (6h6m56.822360364s) [upload=4860]: Got next segment. Inline: false Jul 19 04:57:50.449828921 (6h6m56.822408276s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:57:50.449833700 (6h6m56.822413055s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:57:50.682857520 (6h6m57.055436875s) [seg_pos={0 0}, upload=4860]: Begin upload segment... Jul 19 04:57:50.682898699 (6h6m57.055478054s) [seg_pos={0 0}, upload=4860]: Done begin upload segment. Jul 19 04:57:50.682913377 (6h6m57.055492732s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682920360 (6h6m57.055499705s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682990905 (6h6m57.05557026s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683253236 (6h6m57.055832591s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683500520 (6h6m57.056079875s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683736031 (6h6m57.056315376s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684122380 (6h6m57.056701725s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684336981 (6h6m57.056916336s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684555689 (6h6m57.057135044s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684820506 (6h6m57.057399861s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685013646 (6h6m57.057593001s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685194391 (6h6m57.057773746s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685376059 (6h6m57.057955414s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685559630 (6h6m57.058138985s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685741949 (6h6m57.058321304s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685953323 (6h6m57.058532678s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:52.382568857 (6h6m58.755148202s) [upload=4860]: Next returned nil segment Jul 19 04:57:52.382588915 (6h6m58.75516827s) [upload=4860]: Waiting for error group managing segments... Jul 19 04:57:52.712870771 (6h6m59.085450116s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.713384304 (6h6m59.085963649s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.719689228 (6h6m59.092268583s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.723321257 (6h6m59.095900612s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.797624770 (6h6m59.170204115s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.798066866 (6h6m59.170646221s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.820454393 (6h6m59.193033738s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.864818369 (6h6m59.237397724s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.873179036 (6h6m59.245758391s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.881637702 (6h6m59.254217047s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.884454370 (6h6m59.257033725s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:54.501916491 (6h7m0.874495846s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391729500 (6h7m25.764308845s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391756702 (6h7m25.764336047s) [seg_pos={0 0}, upload=4860]: Segment reached optimal threshold of 13 pieces. Jul 19 04:58:19.391845321 (6h7m25.764424676s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Jul 19 04:58:19.392247070 (6h7m25.764826425s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection Jul 19 04:58:19.392314228 (6h7m25.764893583s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection Jul 19 04:58:19.392340238 (6h7m25.764919593s) [seg_pos={0 0}, upload=4860]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:58:19.392427785 (6h7m25.76500714s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:58:19.392432124 (6h7m25.76501148s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:58:19.702762764 (6h7m26.075342109s) [upload=4860]: Done uploading segments... Jul 19 04:58:19.702781199 (6h7m26.075360554s) [upload=4860]: Upload finished. err: <nil> Duration: 7.497537722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.002314197s FAIL (2025-07-19 05:01:20 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 6.002342491s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................XX...X...X....X..X...] duration: 9.541475166s FAIL (2025-07-19 05:01:06 UTC) orbiter warning: ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== :1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: read tcp 188.166.225.170:50230->77.90.31.19:4841: use of closed network connection; context canceled Jul 19 04:56:54.832707046 (6h6m1.205286401s) [seg_pos={0 0}, upload=4846]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:56:54.832779073 (6h6m1.205358428s) [upload=4846]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:56:54.832783182 (6h6m1.205362527s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:56:55.133341022 (6h6m1.505920377s) [upload=4846]: Done uploading segments... Jul 19 04:56:55.133358805 (6h6m1.50593816s) [upload=4846]: Upload finished. err: <nil> Jul 19 04:57:50.449535830 (6h6m56.822115185s) [upload=4860]: Starting upload Jul 19 04:57:50.449598590 (6h6m56.822177935s) [upload=4860]: Done starting upload Jul 19 04:57:50.449668684 (6h6m56.822248039s) [upload=4860]: Uploading segments... Jul 19 04:57:50.449781009 (6h6m56.822360364s) [upload=4860]: Got next segment. Inline: false Jul 19 04:57:50.449828921 (6h6m56.822408276s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:57:50.449833700 (6h6m56.822413055s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:57:50.682857520 (6h6m57.055436875s) [seg_pos={0 0}, upload=4860]: Begin upload segment... Jul 19 04:57:50.682898699 (6h6m57.055478054s) [seg_pos={0 0}, upload=4860]: Done begin upload segment. Jul 19 04:57:50.682913377 (6h6m57.055492732s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682920360 (6h6m57.055499705s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682990905 (6h6m57.05557026s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683253236 (6h6m57.055832591s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683500520 (6h6m57.056079875s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683736031 (6h6m57.056315376s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684122380 (6h6m57.056701725s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684336981 (6h6m57.056916336s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684555689 (6h6m57.057135044s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684820506 (6h6m57.057399861s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685013646 (6h6m57.057593001s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685194391 (6h6m57.057773746s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685376059 (6h6m57.057955414s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685559630 (6h6m57.058138985s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685741949 (6h6m57.058321304s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685953323 (6h6m57.058532678s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:52.382568857 (6h6m58.755148202s) [upload=4860]: Next returned nil segment Jul 19 04:57:52.382588915 (6h6m58.75516827s) [upload=4860]: Waiting for error group managing segments... Jul 19 04:57:52.712870771 (6h6m59.085450116s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.713384304 (6h6m59.085963649s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.719689228 (6h6m59.092268583s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.723321257 (6h6m59.095900612s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.797624770 (6h6m59.170204115s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.798066866 (6h6m59.170646221s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.820454393 (6h6m59.193033738s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.864818369 (6h6m59.237397724s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.873179036 (6h6m59.245758391s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.881637702 (6h6m59.254217047s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.884454370 (6h6m59.257033725s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:54.501916491 (6h7m0.874495846s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391729500 (6h7m25.764308845s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391756702 (6h7m25.764336047s) [seg_pos={0 0}, upload=4860]: Segment reached optimal threshold of 13 pieces. Jul 19 04:58:19.391845321 (6h7m25.764424676s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Jul 19 04:58:19.392247070 (6h7m25.764826425s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection Jul 19 04:58:19.392314228 (6h7m25.764893583s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection Jul 19 04:58:19.392340238 (6h7m25.764919593s) [seg_pos={0 0}, upload=4860]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:58:19.392427785 (6h7m25.76500714s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:58:19.392432124 (6h7m25.76501148s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:58:19.702762764 (6h7m26.075342109s) [upload=4860]: Done uploading segments... Jul 19 04:58:19.702781199 (6h7m26.075360554s) [upload=4860]: Upload finished. err: <nil> Duration: 9.541497009s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 7.561934261s FAIL (2025-07-19 05:01:02 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 7.561964378s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.050509202s FAIL (2025-07-19 05:00:35 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 6.050537456s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................XX...X...X....X..X..] duration: 5.948090593s FAIL (2025-07-19 04:59:50 UTC) orbiter warning: ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== :1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: read tcp 188.166.225.170:50230->77.90.31.19:4841: use of closed network connection; context canceled Jul 19 04:56:54.832707046 (6h6m1.205286401s) [seg_pos={0 0}, upload=4846]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:56:54.832779073 (6h6m1.205358428s) [upload=4846]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:56:54.832783182 (6h6m1.205362527s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:56:55.133341022 (6h6m1.505920377s) [upload=4846]: Done uploading segments... Jul 19 04:56:55.133358805 (6h6m1.50593816s) [upload=4846]: Upload finished. err: <nil> Jul 19 04:57:50.449535830 (6h6m56.822115185s) [upload=4860]: Starting upload Jul 19 04:57:50.449598590 (6h6m56.822177935s) [upload=4860]: Done starting upload Jul 19 04:57:50.449668684 (6h6m56.822248039s) [upload=4860]: Uploading segments... Jul 19 04:57:50.449781009 (6h6m56.822360364s) [upload=4860]: Got next segment. Inline: false Jul 19 04:57:50.449828921 (6h6m56.822408276s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:57:50.449833700 (6h6m56.822413055s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:57:50.682857520 (6h6m57.055436875s) [seg_pos={0 0}, upload=4860]: Begin upload segment... Jul 19 04:57:50.682898699 (6h6m57.055478054s) [seg_pos={0 0}, upload=4860]: Done begin upload segment. Jul 19 04:57:50.682913377 (6h6m57.055492732s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682920360 (6h6m57.055499705s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.682990905 (6h6m57.05557026s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683253236 (6h6m57.055832591s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683500520 (6h6m57.056079875s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.683736031 (6h6m57.056315376s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684122380 (6h6m57.056701725s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684336981 (6h6m57.056916336s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684555689 (6h6m57.057135044s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.684820506 (6h6m57.057399861s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685013646 (6h6m57.057593001s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685194391 (6h6m57.057773746s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685376059 (6h6m57.057955414s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685559630 (6h6m57.058138985s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685741949 (6h6m57.058321304s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:50.685953323 (6h6m57.058532678s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Uploading piece... Jul 19 04:57:52.382568857 (6h6m58.755148202s) [upload=4860]: Next returned nil segment Jul 19 04:57:52.382588915 (6h6m58.75516827s) [upload=4860]: Waiting for error group managing segments... Jul 19 04:57:52.712870771 (6h6m59.085450116s) [noise=true , address=45.91.116.5:4298 , piece_id=AUGUELVJFWXEPSOEWRCGZM6X4TI2JOUQNYP6WP4VL25ZC7FAPRTQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.713384304 (6h6m59.085963649s) [noise=true , address=45.91.116.9:4263 , piece_id=CPEMTRRYLTUFXTAHXB7P2THNI7T5LZCFO26XPIGATVNH2UWQH6PA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.719689228 (6h6m59.092268583s) [noise=true , address=104.36.82.5:4839 , piece_id=W3C5L2U6NK2DYJSOIFNC2PF2XFXEOHAAWABP7UA3KM7LIXAWSPSQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.723321257 (6h6m59.095900612s) [noise=true , address=104.36.82.10:4077 , piece_id=ZCLW6MI23GWBBSZVV44LHDG4BBLKB656IJCNCVC2PGGVFJK4KV6Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.797624770 (6h6m59.170204115s) [noise=true , address=45.91.116.6:4974 , piece_id=2IEHPCFHSH7LGYE4M6UER42T22OPGFVWEGGQ6CIZ2FZ2HP3YAZWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.798066866 (6h6m59.170646221s) [noise=true , address=45.91.116.4:4938 , piece_id=REUW3UKOOZX56BI3V5ALKCID53EIRRDBMSA2BHCLMQSZGTYJGMDA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.820454393 (6h6m59.193033738s) [noise=true , address=104.36.82.16:4294 , piece_id=4HRMC3IOCWF5I252RH6JTAANPYLX4QFM3UCKXOD67BTAUJ6EDOCQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.864818369 (6h6m59.237397724s) [noise=true , address=45.91.116.10:4081 , piece_id=IN6OJGRO2YDSGNIY3ZNTYTVYLEWKHFKOT24PADM7K7YQNXK5BTEA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.873179036 (6h6m59.245758391s) [noise=true , address=45.91.116.9:4261 , piece_id=6GLY2NCWLS5X6IHCVYB76Z4F2UJ47GXH5HJ6BUBUU7DVW5BT5I5Q, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.881637702 (6h6m59.254217047s) [noise=true , address=45.91.116.5:4299 , piece_id=RPRYWZHCNVP72ZBR4BGMCOI245RG7ZEIWKVCNX3HRZVODGJIM26A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:52.884454370 (6h6m59.257033725s) [noise=true , address=104.36.82.3:4767 , piece_id=MNQW4WEQRFJQXUKITTVSOQN4J3T7AEXPYN37Z6OMFLFOASRQKZYA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:57:54.501916491 (6h7m0.874495846s) [noise=true , address=45.91.116.7:4011 , piece_id=LI35TO3RUCTEYICC7HUI6P34Z2KJH3S43L2GRAE646FAVXNHRXWA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391729500 (6h7m25.764308845s) [noise=true , address=77.90.31.11:4553 , piece_id=ECHSCGV2FSRCWW5M4QKGZMCQ4AHFCIUFBWPRURFCPFJR5YUOC42A, seg_pos={0 0}, upload=4860]: Done uploading piece. err: <nil> Jul 19 04:58:19.391756702 (6h7m25.764336047s) [seg_pos={0 0}, upload=4860]: Segment reached optimal threshold of 13 pieces. Jul 19 04:58:19.391845321 (6h7m25.764424676s) [noise=true , address=77.90.31.14:4661 , piece_id=X6UIHHUTOKC6R3BIJXZETTR4EBDJ4EHQYCYJVG3KLSAPVMZZMADA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Jul 19 04:58:19.392247070 (6h7m25.764826425s) [noise=true , address=77.90.31.18:4806 , piece_id=MYZZEPG546S5I67HCIBSNMTK2UGT7XOBMTGYPTPZAIMNLA6TWAKA, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45822->77.90.31.18:4806: use of closed network connection Jul 19 04:58:19.392314228 (6h7m25.764893583s) [noise=true , address=77.90.31.17:4771 , piece_id=GQQ5R52KDOOI5WOXKTJKZXTIECNU3QWDM4KLGOR7UMORDPMHJUJQ, seg_pos={0 0}, upload=4860]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46240->77.90.31.17:4771: use of closed network connection Jul 19 04:58:19.392340238 (6h7m25.764919593s) [seg_pos={0 0}, upload=4860]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:58:19.392427785 (6h7m25.76500714s) [upload=4860]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:58:19.392432124 (6h7m25.76501148s) [upload=4860]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:58:19.702762764 (6h7m26.075342109s) [upload=4860]: Done uploading segments... Jul 19 04:58:19.702781199 (6h7m26.075360554s) [upload=4860]: Upload finished. err: <nil> Duration: 5.948114849s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)