Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.521845272s FAIL (2025-07-19 04:53:32 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 5.521869628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XX...X...X..] duration: 5.731515236s FAIL (2025-07-19 04:53:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== rue , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.812867503 (6h0m44.185446858s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813060833 (6h0m44.185640188s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813337573 (6h0m44.185916929s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813378882 (6h0m44.185958237s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813633609 (6h0m44.186212964s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813830185 (6h0m44.18640954s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813920779 (6h0m44.186500124s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814123376 (6h0m44.186702731s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814174234 (6h0m44.186753589s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814414464 (6h0m44.186993819s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814426847 (6h0m44.187006202s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814640716 (6h0m44.187220071s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814715981 (6h0m44.187295336s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814875566 (6h0m44.187454911s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:39.567334189 (6h0m45.939913544s) [upload=4776]: Next returned nil segment Jul 19 04:51:39.567352354 (6h0m45.939931709s) [upload=4776]: Waiting for error group managing segments... Jul 19 04:51:39.882055541 (6h0m46.254634886s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:39.950840033 (6h0m46.323419388s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:40.030269575 (6h0m46.40284894s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:52:06.886171942 (6h1m13.258751347s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: rpc: dial tcp 77.90.31.11:4553: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886226055 (6h1m13.25880541s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886263507 (6h1m13.258842852s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886300778 (6h1m13.258880143s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886328180 (6h1m13.258907585s) [noise=true , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886392123 (6h1m13.258971478s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 188.166.225.170:48642->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886404346 (6h1m13.258983701s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:49268->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886435656 (6h1m13.259015011s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886453570 (6h1m13.259032925s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886473689 (6h1m13.259053044s) [noise=true , address=77.90.31.18:4807 , piece_id=NICW3OGWT627257ZXPRSC3ASCUEWDSX6FKRMPKUKGO2OUVPUFZJA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: rpc: dial tcp 77.90.31.18:4807: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886487124 (6h1m13.259066469s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886505169 (6h1m13.259084514s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886518144 (6h1m13.259097499s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:51384->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886540957 (6h1m13.259120322s) [seg_pos={0 0}, upload=4776]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Jul 19 04:52:07.115640319 (6h1m13.488219674s) [upload=4776]: Done uploading segments... Jul 19 04:52:07.115664435 (6h1m13.48824379s) [upload=4776]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 5.731544912s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX...X...X.] duration: 25.46666128s FAIL (2025-07-19 04:52:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== rue , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.812867503 (6h0m44.185446858s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813060833 (6h0m44.185640188s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813337573 (6h0m44.185916929s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813378882 (6h0m44.185958237s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813633609 (6h0m44.186212964s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813830185 (6h0m44.18640954s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813920779 (6h0m44.186500124s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814123376 (6h0m44.186702731s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814174234 (6h0m44.186753589s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814414464 (6h0m44.186993819s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814426847 (6h0m44.187006202s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814640716 (6h0m44.187220071s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814715981 (6h0m44.187295336s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814875566 (6h0m44.187454911s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:39.567334189 (6h0m45.939913544s) [upload=4776]: Next returned nil segment Jul 19 04:51:39.567352354 (6h0m45.939931709s) [upload=4776]: Waiting for error group managing segments... Jul 19 04:51:39.882055541 (6h0m46.254634886s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:39.950840033 (6h0m46.323419388s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:40.030269575 (6h0m46.40284894s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:52:06.886171942 (6h1m13.258751347s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: rpc: dial tcp 77.90.31.11:4553: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886226055 (6h1m13.25880541s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886263507 (6h1m13.258842852s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886300778 (6h1m13.258880143s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886328180 (6h1m13.258907585s) [noise=true , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886392123 (6h1m13.258971478s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 188.166.225.170:48642->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886404346 (6h1m13.258983701s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:49268->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886435656 (6h1m13.259015011s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886453570 (6h1m13.259032925s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886473689 (6h1m13.259053044s) [noise=true , address=77.90.31.18:4807 , piece_id=NICW3OGWT627257ZXPRSC3ASCUEWDSX6FKRMPKUKGO2OUVPUFZJA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: rpc: dial tcp 77.90.31.18:4807: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886487124 (6h1m13.259066469s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886505169 (6h1m13.259084514s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886518144 (6h1m13.259097499s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:51384->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886540957 (6h1m13.259120322s) [seg_pos={0 0}, upload=4776]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Jul 19 04:52:07.115640319 (6h1m13.488219674s) [upload=4776]: Done uploading segments... Jul 19 04:52:07.115664435 (6h1m13.48824379s) [upload=4776]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 25.466685015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 17.65402707s FAIL (2025-07-19 04:52:35 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 17.654052651s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.002767912s FAIL (2025-07-19 04:52:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4855s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:51:09.109709627 (6h0m15.482288982s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:51:09.556145753 (6h0m15.928725108s) [seg_pos={0 0}, upload=4769]: Begin upload segment... Jul 19 04:51:09.556194817 (6h0m15.928774172s) [seg_pos={0 0}, upload=4769]: Done begin upload segment. Jul 19 04:51:09.556223893 (6h0m15.928803248s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556252197 (6h0m15.928831552s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556579383 (6h0m15.929158738s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.556854469 (6h0m15.929433824s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557094107 (6h0m15.929673462s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557336381 (6h0m15.929915736s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557557885 (6h0m15.93013724s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.557762296 (6h0m15.930341651s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558014689 (6h0m15.930594044s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558215974 (6h0m15.930795319s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558409364 (6h0m15.930988719s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558623504 (6h0m15.931202859s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558805162 (6h0m15.931384517s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.558999373 (6h0m15.931578728s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559184768 (6h0m15.931764123s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559368719 (6h0m15.931948074s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559549706 (6h0m15.932129061s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:09.559727836 (6h0m15.932307191s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Uploading piece... Jul 19 04:51:10.663359087 (6h0m17.035938442s) [upload=4769]: Next returned nil segment Jul 19 04:51:10.663378664 (6h0m17.035958019s) [upload=4769]: Waiting for error group managing segments... Jul 19 04:51:10.962945479 (6h0m17.335524834s) [noise=true , address=79.127.205.233:4622 , piece_id=6WWNYQTH5MGAJOEB7BXKZDJOMCAZXDW66J3OEANGDNJDNUMLZM7Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.967689807 (6h0m17.340269152s) [noise=true , address=79.127.205.235:4660 , piece_id=UOUOGE2VP7PX7EMRZLPMCWFCYEZERM3DQMCQ6NG4J2K2DEIZR63Q, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:10.976968631 (6h0m17.349547986s) [noise=true , address=64.49.13.138:4482 , piece_id=MAW2M4US2OQNOS3WI67AA7NIBTJDE5T2JQTXKFPBLGVGXKVCFGTQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.048750700 (6h0m17.421330055s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBOZI3UGTQXAQBLZ2F45Z3OEYL65F5PQBXRF2DQMZ7NGF7VDRPA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.221642403 (6h0m17.594221758s) [noise=true , address=79.127.220.98:4432 , piece_id=2SUTFQ7FVDPAM2H4W3OLW6AVH3VJ33IJFE4L52SA3ET5LGBMOMPQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.317070435 (6h0m17.68964979s) [noise=true , address=45.91.95.4:4011 , piece_id=ZMBAS6KKHYVIRE6OBL2NFVCWMM4TKQ7J64BD3SYXIK54XCEDI2XA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.324002660 (6h0m17.696582015s) [noise=true , address=45.91.95.5:4019 , piece_id=YTSBCN2LPHEMRRAZIHJGVB7NBOTRXVEHD2TWUCTTLBNFFYEWWDKA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.460976317 (6h0m17.833555672s) [noise=true , address=79.127.201.215:4357 , piece_id=75XWAG2VQWEYCWN2Q3WRIZSBI7LDOISZJPXQLQ2GIPNEN34VLSUQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.462584603 (6h0m17.835163958s) [noise=true , address=45.91.116.6:4007 , piece_id=A2AFVTZBXSGBOE7XIM3B5HRN4STITFBXO6MZRJ365URNAAOZ3PSQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.490392973 (6h0m17.862972328s) [noise=true , address=45.91.116.9:4264 , piece_id=GHM5BKZKMT2QOHHBOS4FSPOWEWSFQRGRVVR4GSNDPHVIFRZHQL2A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.502040780 (6h0m17.874620135s) [noise=true , address=45.91.116.14:4219 , piece_id=PDW6JPRHCVMHHOWIHT6JKV4CTSC3JY6NDKDVFZSO47OYNPZ7PGGA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.600434096 (6h0m17.973013451s) [noise=true , address=79.127.219.51:4211 , piece_id=HO5PLWBVU5YAYNYMN352NA73SM4WBJV4WZ7XLSQTCQ76UAL2TOJA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.609008984 (6h0m17.981588339s) [noise=true , address=104.36.82.19:4423 , piece_id=PIJUHDUND36HQL67I33D7A3EULMJHS7N55MN4OFQJ4OWNQ42O47A, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620825273 (6h0m17.993404628s) [noise=true , address=79.127.219.38:4023 , piece_id=PTPOZGLYV4MZTOFZNN3NCGYL7DFAST4PXKEUIVTMFL7HV4GZTCLQ, seg_pos={0 0}, upload=4769]: Done uploading piece. err: <nil> Jul 19 04:51:11.620838659 (6h0m17.993418014s) [seg_pos={0 0}, upload=4769]: Segment reached optimal threshold of 14 pieces. Jul 19 04:51:11.621067807 (6h0m17.993647163s) [noise=true , address=77.90.31.11:4564 , piece_id=L2U7JFJRDQOFA2AYVAGTI4ZW4VABCXKMXWBIUWQO2QNNUUBJ74BA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1pKhKeEpeFSQDeKBUkSwGp58G9nAcdQZ19zuuuae522vMv7AP9): protocol: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45814->77.90.31.11:4564: use of closed network connection Jul 19 04:51:11.621134234 (6h0m17.993713589s) [noise=true , address=77.90.31.10:4546 , piece_id=R6LCA5WZOEBDURL5S3MEK22YPVYXX6EXSS4J4W57543HELYXOKZA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jjRZPZ6TDPepFAmvv81dREkDJDrXqeu9wqzNYkeaGemFFdxwK): protocol: read tcp 188.166.225.170:58476->77.90.31.10:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621174191 (6h0m17.993753546s) [noise=true , address=77.90.31.16:4768 , piece_id=YNLSONFM6COZEVF6ZNQR2EA6V4LSQFZJONIXR4AFTNNMFGPV57JA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UchewmtxBrtSZSpVAJFdZmcTsQVQxkjK9re6cgUGqDhyonQeF): protocol: read tcp 188.166.225.170:56326->77.90.31.16:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621216842 (6h0m17.993796197s) [noise=true , address=77.90.31.18:4813 , piece_id=VDOT7BB4O7XYJHW6BFTAYQZZYLZU5PAUXVJGSXJ7FDI4TH425COA, seg_pos={0 0}, upload=4769]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5): protocol: read tcp 188.166.225.170:49704->77.90.31.18:4813: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:51:11.621234917 (6h0m17.993814272s) [seg_pos={0 0}, upload=4769]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Jul 19 04:51:11.621301444 (6h0m17.993880799s) [upload=4769]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:51:11.621305402 (6h0m17.993884758s) [upload=4769]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:51:11.923136970 (6h0m18.295716325s) [upload=4769]: Done uploading segments... Jul 19 04:51:11.923156627 (6h0m18.295735972s) [upload=4769]: Upload finished. err: <nil> Duration: 1m0.002786519s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XX...X...X] duration: 30.231442823s FAIL (2025-07-19 04:52:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== rue , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.812867503 (6h0m44.185446858s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813060833 (6h0m44.185640188s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813337573 (6h0m44.185916929s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813378882 (6h0m44.185958237s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813633609 (6h0m44.186212964s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813830185 (6h0m44.18640954s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.813920779 (6h0m44.186500124s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814123376 (6h0m44.186702731s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814174234 (6h0m44.186753589s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814414464 (6h0m44.186993819s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814426847 (6h0m44.187006202s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814640716 (6h0m44.187220071s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814715981 (6h0m44.187295336s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:37.814875566 (6h0m44.187454911s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Uploading piece... Jul 19 04:51:39.567334189 (6h0m45.939913544s) [upload=4776]: Next returned nil segment Jul 19 04:51:39.567352354 (6h0m45.939931709s) [upload=4776]: Waiting for error group managing segments... Jul 19 04:51:39.882055541 (6h0m46.254634886s) [noise=true , address=104.36.82.7:4911 , piece_id=BXSTWCNP6O2R3OIBHMEPZW4DMGYIUP4KGJXVQWWQ3HBXWREE7KHA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:39.950840033 (6h0m46.323419388s) [noise=true , address=104.36.82.11:4113 , piece_id=J47Y3LBFN7N5RENSKXBBE64HEHJLZS4U273HJXCZBABCZLYRU4OQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:51:40.030269575 (6h0m46.40284894s) [noise=true , address=104.36.82.5:4839 , piece_id=HE7RLNECHYYDVXFIHKMWF32OUWM2TIV5OTFFX3WEZDAFBWYHU4TQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: <nil> Jul 19 04:52:06.886171942 (6h1m13.258751347s) [noise=true , address=77.90.31.11:4553 , piece_id=3KUTEWRXBHVGFCA6AXRRR3FTJYQEE7QOW6UFVUSUFMJ2L6527GXQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: rpc: dial tcp 77.90.31.11:4553: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886226055 (6h1m13.25880541s) [noise=true , address=45.91.116.10:4082 , piece_id=SCUONBPQGCDONRQFZCUJ256C7AKWUP43TTYYFBTCVS6SHHEIKWKA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886263507 (6h1m13.258842852s) [noise=true , address=77.90.31.10:4520 , piece_id=HWHBZ3ZIRQNLW7IPML5O3CJOHHZWCZQWWYFRKKGPFEKA2PYMOONA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: rpc: dial tcp 77.90.31.10:4520: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886300778 (6h1m13.258880143s) [noise=true , address=45.91.116.12:4225 , piece_id=GLJL3AAISDRWMWX5ULJCKAEGACS5YHWNMZNTEYMHCTCWR6TLEEEQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886328180 (6h1m13.258907585s) [noise=true , address=45.91.116.11:4119 , piece_id=KTLVVJP2QCMK4KRWZ5SE2KD5GUCA55Z4LSEJ5K4DQ7XPVWKEVKAQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: rpc: dial tcp 45.91.116.11:4119: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886392123 (6h1m13.258971478s) [noise=true , address=45.91.116.6:4974 , piece_id=VE6BT2CDEE36HVWQYUBLEITHUHI5IR223PE5XP5FI7YAYPO7MZKQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: write tcp 188.166.225.170:48642->45.91.116.6:4974: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886404346 (6h1m13.258983701s) [noise=true , address=77.90.31.17:4772 , piece_id=HJY5L2HO3OG3M5TDOAHW6MWPEGMJ4NJXEYGIORMV6D5Q4R5POHLQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:49268->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886435656 (6h1m13.259015011s) [noise=true , address=45.91.116.4:4938 , piece_id=TRHEJKBA3OWNONXLKBFFMGPCQFCWTB2XBE7WSQOITUPGMC7HG5IQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886453570 (6h1m13.259032925s) [noise=true , address=45.91.116.8:4045 , piece_id=Y2LNSNTKMH2QAMO4J6AQWQW7TWXC4QAW7H2VATFT2ENXYNCIGYMQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886473689 (6h1m13.259053044s) [noise=true , address=77.90.31.18:4807 , piece_id=NICW3OGWT627257ZXPRSC3ASCUEWDSX6FKRMPKUKGO2OUVPUFZJA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: rpc: dial tcp 77.90.31.18:4807: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886487124 (6h1m13.259066469s) [noise=true , address=45.91.116.6:4973 , piece_id=2SL6JAYP4LTT5RKTUM34T4T2QG5JX3YVTIH3LLBCRJD2MJRHUJZA, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886505169 (6h1m13.259084514s) [noise=true , address=45.91.116.5:4299 , piece_id=WMPIHWASRJ4B74WDUWAOTMEKO3CFOY4BUVC43AX5XVM47PCG6UDQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886518144 (6h1m13.259097499s) [noise=true , address=45.91.116.9:4262 , piece_id=KMPD4V4P4JAR4ZGADXMGQVR4Y7K67XC4DEFN5XN27XCVL3HQOHNQ, seg_pos={0 0}, upload=4776]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: write tcp 188.166.225.170:51384->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 04:52:06.886540957 (6h1m13.259120322s) [seg_pos={0 0}, upload=4776]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Jul 19 04:52:07.115640319 (6h1m13.488219674s) [upload=4776]: Done uploading segments... Jul 19 04:52:07.115664435 (6h1m13.48824379s) [upload=4776]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.231467181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XX...X...] duration: 16.041941359s FAIL (2025-07-19 04:50:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== piecestore close: write tcp 188.166.225.170:39986->45.91.116.6:4973: use of closed network connection Jul 19 04:48:06.309744850 (5h57m12.682324205s) [seg_pos={0 0}, upload=4729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:48:06.309854891 (5h57m12.682434246s) [upload=4729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:48:06.309860030 (5h57m12.682439386s) [upload=4729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:48:06.598523061 (5h57m12.971102416s) [upload=4729]: Done uploading segments... Jul 19 04:48:06.598537669 (5h57m12.971117025s) [upload=4729]: Upload finished. err: <nil> Jul 19 04:49:12.752251238 (5h58m19.124830583s) [upload=4745]: Starting upload Jul 19 04:49:12.752305712 (5h58m19.124885067s) [upload=4745]: Done starting upload Jul 19 04:49:12.752365095 (5h58m19.12494445s) [upload=4745]: Uploading segments... Jul 19 04:49:12.752483102 (5h58m19.125062457s) [upload=4745]: Got next segment. Inline: false Jul 19 04:49:12.752510584 (5h58m19.125089939s) [upload=4745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:49:12.752515223 (5h58m19.125094578s) [upload=4745]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:49:13.010638919 (5h58m19.383218275s) [seg_pos={0 0}, upload=4745]: Begin upload segment... Jul 19 04:49:13.010675340 (5h58m19.383254685s) [seg_pos={0 0}, upload=4745]: Done begin upload segment. Jul 19 04:49:13.010696800 (5h58m19.383276155s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.010748920 (5h58m19.383328275s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011046149 (5h58m19.383625504s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011303110 (5h58m19.383882455s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011520306 (5h58m19.384099661s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011587024 (5h58m19.384166379s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011884954 (5h58m19.384464309s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011952944 (5h58m19.384532299s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012231887 (5h58m19.384811242s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012244302 (5h58m19.384823647s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012466617 (5h58m19.385045972s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012516673 (5h58m19.385096028s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012684614 (5h58m19.385263969s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012783643 (5h58m19.385362998s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012944491 (5h58m19.385523846s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.013030355 (5h58m19.38560971s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:14.789481216 (5h58m21.162060581s) [upload=4745]: Next returned nil segment Jul 19 04:49:14.789501124 (5h58m21.162080479s) [upload=4745]: Waiting for error group managing segments... Jul 19 04:49:15.071545521 (5h58m21.444124876s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.134660315 (5h58m21.50723967s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.137679563 (5h58m21.510258908s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.148916173 (5h58m21.521495518s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.157699109 (5h58m21.530278464s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.160473116 (5h58m21.533052471s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.173032427 (5h58m21.545611772s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:16.715308675 (5h58m23.08788803s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:18.769949176 (5h58m25.142528531s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:24.060312016 (5h58m30.432891371s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:40.433306047 (5h58m46.805885402s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:42.097235347 (5h58m48.469814802s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097317364 (5h58m48.469896719s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097460398 (5h58m48.470039753s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097547455 (5h58m48.47012681s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection Jul 19 04:49:42.097583073 (5h58m48.470162428s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection Jul 19 04:49:42.097595536 (5h58m48.470174891s) [seg_pos={0 0}, upload=4745]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 04:49:42.328174730 (5h58m48.700754075s) [upload=4745]: Done uploading segments... Jul 19 04:49:42.328198956 (5h58m48.700778301s) [upload=4745]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 16.041969604s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XX...X..] duration: 22.733735232s FAIL (2025-07-19 04:50:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== piecestore close: write tcp 188.166.225.170:39986->45.91.116.6:4973: use of closed network connection Jul 19 04:48:06.309744850 (5h57m12.682324205s) [seg_pos={0 0}, upload=4729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:48:06.309854891 (5h57m12.682434246s) [upload=4729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:48:06.309860030 (5h57m12.682439386s) [upload=4729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:48:06.598523061 (5h57m12.971102416s) [upload=4729]: Done uploading segments... Jul 19 04:48:06.598537669 (5h57m12.971117025s) [upload=4729]: Upload finished. err: <nil> Jul 19 04:49:12.752251238 (5h58m19.124830583s) [upload=4745]: Starting upload Jul 19 04:49:12.752305712 (5h58m19.124885067s) [upload=4745]: Done starting upload Jul 19 04:49:12.752365095 (5h58m19.12494445s) [upload=4745]: Uploading segments... Jul 19 04:49:12.752483102 (5h58m19.125062457s) [upload=4745]: Got next segment. Inline: false Jul 19 04:49:12.752510584 (5h58m19.125089939s) [upload=4745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:49:12.752515223 (5h58m19.125094578s) [upload=4745]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:49:13.010638919 (5h58m19.383218275s) [seg_pos={0 0}, upload=4745]: Begin upload segment... Jul 19 04:49:13.010675340 (5h58m19.383254685s) [seg_pos={0 0}, upload=4745]: Done begin upload segment. Jul 19 04:49:13.010696800 (5h58m19.383276155s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.010748920 (5h58m19.383328275s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011046149 (5h58m19.383625504s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011303110 (5h58m19.383882455s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011520306 (5h58m19.384099661s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011587024 (5h58m19.384166379s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011884954 (5h58m19.384464309s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011952944 (5h58m19.384532299s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012231887 (5h58m19.384811242s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012244302 (5h58m19.384823647s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012466617 (5h58m19.385045972s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012516673 (5h58m19.385096028s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012684614 (5h58m19.385263969s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012783643 (5h58m19.385362998s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012944491 (5h58m19.385523846s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.013030355 (5h58m19.38560971s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:14.789481216 (5h58m21.162060581s) [upload=4745]: Next returned nil segment Jul 19 04:49:14.789501124 (5h58m21.162080479s) [upload=4745]: Waiting for error group managing segments... Jul 19 04:49:15.071545521 (5h58m21.444124876s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.134660315 (5h58m21.50723967s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.137679563 (5h58m21.510258908s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.148916173 (5h58m21.521495518s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.157699109 (5h58m21.530278464s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.160473116 (5h58m21.533052471s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.173032427 (5h58m21.545611772s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:16.715308675 (5h58m23.08788803s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:18.769949176 (5h58m25.142528531s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:24.060312016 (5h58m30.432891371s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:40.433306047 (5h58m46.805885402s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:42.097235347 (5h58m48.469814802s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097317364 (5h58m48.469896719s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097460398 (5h58m48.470039753s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097547455 (5h58m48.47012681s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection Jul 19 04:49:42.097583073 (5h58m48.470162428s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection Jul 19 04:49:42.097595536 (5h58m48.470174891s) [seg_pos={0 0}, upload=4745]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 04:49:42.328174730 (5h58m48.700754075s) [upload=4745]: Done uploading segments... Jul 19 04:49:42.328198956 (5h58m48.700778301s) [upload=4745]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 22.733761994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XX...X.] duration: 11.415521568s FAIL (2025-07-19 04:49:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== piecestore close: write tcp 188.166.225.170:39986->45.91.116.6:4973: use of closed network connection Jul 19 04:48:06.309744850 (5h57m12.682324205s) [seg_pos={0 0}, upload=4729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:48:06.309854891 (5h57m12.682434246s) [upload=4729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:48:06.309860030 (5h57m12.682439386s) [upload=4729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:48:06.598523061 (5h57m12.971102416s) [upload=4729]: Done uploading segments... Jul 19 04:48:06.598537669 (5h57m12.971117025s) [upload=4729]: Upload finished. err: <nil> Jul 19 04:49:12.752251238 (5h58m19.124830583s) [upload=4745]: Starting upload Jul 19 04:49:12.752305712 (5h58m19.124885067s) [upload=4745]: Done starting upload Jul 19 04:49:12.752365095 (5h58m19.12494445s) [upload=4745]: Uploading segments... Jul 19 04:49:12.752483102 (5h58m19.125062457s) [upload=4745]: Got next segment. Inline: false Jul 19 04:49:12.752510584 (5h58m19.125089939s) [upload=4745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:49:12.752515223 (5h58m19.125094578s) [upload=4745]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:49:13.010638919 (5h58m19.383218275s) [seg_pos={0 0}, upload=4745]: Begin upload segment... Jul 19 04:49:13.010675340 (5h58m19.383254685s) [seg_pos={0 0}, upload=4745]: Done begin upload segment. Jul 19 04:49:13.010696800 (5h58m19.383276155s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.010748920 (5h58m19.383328275s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011046149 (5h58m19.383625504s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011303110 (5h58m19.383882455s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011520306 (5h58m19.384099661s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011587024 (5h58m19.384166379s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011884954 (5h58m19.384464309s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011952944 (5h58m19.384532299s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012231887 (5h58m19.384811242s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012244302 (5h58m19.384823647s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012466617 (5h58m19.385045972s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012516673 (5h58m19.385096028s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012684614 (5h58m19.385263969s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012783643 (5h58m19.385362998s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012944491 (5h58m19.385523846s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.013030355 (5h58m19.38560971s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:14.789481216 (5h58m21.162060581s) [upload=4745]: Next returned nil segment Jul 19 04:49:14.789501124 (5h58m21.162080479s) [upload=4745]: Waiting for error group managing segments... Jul 19 04:49:15.071545521 (5h58m21.444124876s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.134660315 (5h58m21.50723967s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.137679563 (5h58m21.510258908s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.148916173 (5h58m21.521495518s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.157699109 (5h58m21.530278464s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.160473116 (5h58m21.533052471s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.173032427 (5h58m21.545611772s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:16.715308675 (5h58m23.08788803s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:18.769949176 (5h58m25.142528531s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:24.060312016 (5h58m30.432891371s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:40.433306047 (5h58m46.805885402s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:42.097235347 (5h58m48.469814802s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097317364 (5h58m48.469896719s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097460398 (5h58m48.470039753s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097547455 (5h58m48.47012681s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection Jul 19 04:49:42.097583073 (5h58m48.470162428s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection Jul 19 04:49:42.097595536 (5h58m48.470174891s) [seg_pos={0 0}, upload=4745]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 04:49:42.328174730 (5h58m48.700754075s) [upload=4745]: Done uploading segments... Jul 19 04:49:42.328198956 (5h58m48.700778301s) [upload=4745]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 11.415536576s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XX...X] duration: 30.232600724s FAIL (2025-07-19 04:49:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== piecestore close: write tcp 188.166.225.170:39986->45.91.116.6:4973: use of closed network connection Jul 19 04:48:06.309744850 (5h57m12.682324205s) [seg_pos={0 0}, upload=4729]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 04:48:06.309854891 (5h57m12.682434246s) [upload=4729]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 04:48:06.309860030 (5h57m12.682439386s) [upload=4729]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 04:48:06.598523061 (5h57m12.971102416s) [upload=4729]: Done uploading segments... Jul 19 04:48:06.598537669 (5h57m12.971117025s) [upload=4729]: Upload finished. err: <nil> Jul 19 04:49:12.752251238 (5h58m19.124830583s) [upload=4745]: Starting upload Jul 19 04:49:12.752305712 (5h58m19.124885067s) [upload=4745]: Done starting upload Jul 19 04:49:12.752365095 (5h58m19.12494445s) [upload=4745]: Uploading segments... Jul 19 04:49:12.752483102 (5h58m19.125062457s) [upload=4745]: Got next segment. Inline: false Jul 19 04:49:12.752510584 (5h58m19.125089939s) [upload=4745]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:49:12.752515223 (5h58m19.125094578s) [upload=4745]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:49:13.010638919 (5h58m19.383218275s) [seg_pos={0 0}, upload=4745]: Begin upload segment... Jul 19 04:49:13.010675340 (5h58m19.383254685s) [seg_pos={0 0}, upload=4745]: Done begin upload segment. Jul 19 04:49:13.010696800 (5h58m19.383276155s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.010748920 (5h58m19.383328275s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011046149 (5h58m19.383625504s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011303110 (5h58m19.383882455s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011520306 (5h58m19.384099661s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011587024 (5h58m19.384166379s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011884954 (5h58m19.384464309s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.011952944 (5h58m19.384532299s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012231887 (5h58m19.384811242s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012244302 (5h58m19.384823647s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012466617 (5h58m19.385045972s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012516673 (5h58m19.385096028s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012684614 (5h58m19.385263969s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012783643 (5h58m19.385362998s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.012944491 (5h58m19.385523846s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:13.013030355 (5h58m19.38560971s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Uploading piece... Jul 19 04:49:14.789481216 (5h58m21.162060581s) [upload=4745]: Next returned nil segment Jul 19 04:49:14.789501124 (5h58m21.162080479s) [upload=4745]: Waiting for error group managing segments... Jul 19 04:49:15.071545521 (5h58m21.444124876s) [noise=true , address=45.91.116.12:4227 , piece_id=4GPHHTFG32HBCPCOUAJL5C5PIQ6E72YBXJ73G3TNK2HTEEG54BKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.134660315 (5h58m21.50723967s) [noise=true , address=104.36.82.7:4912 , piece_id=I4JWPSUBRTTBPHVPUR3HJYIX3BRO3WF2IU7ZTROMNEXXHXHFT7MQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.137679563 (5h58m21.510258908s) [noise=true , address=104.36.82.4:4804 , piece_id=RXYG54CPRYRAPMCYWEC7TTL3DDKDDXOQE3HBPTUMVUN5OOPEE6DQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.148916173 (5h58m21.521495518s) [noise=true , address=45.91.116.6:4974 , piece_id=HDBVZOH63V7EKXTZ37YBYF7I6HUPS3MI4WSWXFFQ6IOQ7IZO6NKQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.157699109 (5h58m21.530278464s) [noise=true , address=45.91.116.4:4937 , piece_id=WLYMENE36MMYOMDXHFZV3T5MV4VYUMN6ARW6TK5Y4AOZDP7DHJPA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.160473116 (5h58m21.533052471s) [noise=true , address=45.91.116.11:4118 , piece_id=A32CYBEI3W34X4HBJCWZXETYVTOLBBKOBMSZD3WFBFETTEZ4TDBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:15.173032427 (5h58m21.545611772s) [noise=true , address=45.91.116.10:4083 , piece_id=7CW5TM3CBI442NI4BXZSM2ME6CIAIY2E7WDV7KYWC3NL4FAL4GCQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:16.715308675 (5h58m23.08788803s) [noise=true , address=45.91.116.7:4010 , piece_id=SR2F2EBO5FPUNAJYR5F6UUWXK2QUD6QEE34RQHU3CFKTRUYBYHUQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:18.769949176 (5h58m25.142528531s) [noise=true , address=77.90.31.10:4520 , piece_id=XJGANGGBC2KZJFIAXENXCG73PEOJVWYVHICQKL77UV2OA2WHC4OQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:24.060312016 (5h58m30.432891371s) [noise=true , address=77.90.31.17:4771 , piece_id=I3HS7BRM2CIYWFAAINHNS3EQDLLF6PZ5LHRMYQZEHXTDT4H6VWIA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:40.433306047 (5h58m46.805885402s) [noise=true , address=77.90.31.19:4841 , piece_id=QA2H2SFWBQICG4RMYTHYEPPNPTQACT2R6ZNBWM35T2TQ27ANVUPQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: <nil> Jul 19 04:49:42.097235347 (5h58m48.469814802s) [noise=true , address=77.90.31.13:4628 , piece_id=FGE7FLWBYXPPADCEX4PY2O6UWALHC5JLJOVJ3W3TT5EZXDXC7DBQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097317364 (5h58m48.469896719s) [noise=true , address=77.90.31.13:4627 , piece_id=UL2UN6X6AHXHWGLOSWHT5PYO5HAYRC2RVWMGMSJS44MPQXTDLGTQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097460398 (5h58m48.470039753s) [noise=true , address=77.90.31.11:4555 , piece_id=IEVKSH3O4G4DNR6PVHWKZMKAS75Q66PPA5KBFVCGRBVEAHUOCEYQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:49:42.097547455 (5h58m48.47012681s) [noise=true , address=77.90.31.18:4806 , piece_id=4I4CPTLBN6K2W2GXNZHOW7FEBGYVE5YKLIESZX7QNMYAHZORYRHA, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33648->77.90.31.18:4806: use of closed network connection Jul 19 04:49:42.097583073 (5h58m48.470162428s) [noise=true , address=77.90.31.14:4663 , piece_id=NNYRHGEUAUUV3BA7O4GXOBKDFLYTPH2EMBVH43UJZJCVRYF7MOHQ, seg_pos={0 0}, upload=4745]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33918->77.90.31.14:4663: use of closed network connection Jul 19 04:49:42.097595536 (5h58m48.470174891s) [seg_pos={0 0}, upload=4745]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 04:49:42.328174730 (5h58m48.700754075s) [upload=4745]: Done uploading segments... Jul 19 04:49:42.328198956 (5h58m48.700778301s) [upload=4745]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.232620372s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 6.549320112s FAIL (2025-07-19 04:48:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ode:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection Jul 19 04:44:47.415361105 (5h53m53.78794046s) [seg_pos={0 0}, upload=4684]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 19 04:44:47.647064873 (5h53m54.019644228s) [upload=4684]: Done uploading segments... Jul 19 04:44:47.647091534 (5h53m54.019670889s) [upload=4684]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 19 04:45:21.280261101 (5h54m27.652840456s) [upload=4698]: Starting upload Jul 19 04:45:21.280319553 (5h54m27.652898908s) [upload=4698]: Done starting upload Jul 19 04:45:21.280540766 (5h54m27.653120121s) [upload=4698]: Uploading segments... Jul 19 04:45:21.280583858 (5h54m27.653163203s) [upload=4698]: Got next segment. Inline: false Jul 19 04:45:21.280620809 (5h54m27.653200164s) [upload=4698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:45:21.280625227 (5h54m27.653204583s) [upload=4698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:45:21.538947495 (5h54m27.91152685s) [seg_pos={0 0}, upload=4698]: Begin upload segment... Jul 19 04:45:21.539250875 (5h54m27.91183023s) [seg_pos={0 0}, upload=4698]: Done begin upload segment. Jul 19 04:45:21.539279179 (5h54m27.911858534s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540136549 (5h54m27.912715904s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540142360 (5h54m27.912721715s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540514533 (5h54m27.913093888s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540577964 (5h54m27.913157319s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540891614 (5h54m27.913470969s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540943022 (5h54m27.913522377s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541256162 (5h54m27.913835507s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541294124 (5h54m27.913873479s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541642441 (5h54m27.914221786s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541666407 (5h54m27.914245762s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542077954 (5h54m27.914657309s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542476085 (5h54m27.915055441s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543576250 (5h54m27.916155605s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543980244 (5h54m27.916559609s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.544371722 (5h54m27.916951078s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:23.300638583 (5h54m29.673217969s) [upload=4698]: Next returned nil segment Jul 19 04:45:23.300658502 (5h54m29.673237877s) [upload=4698]: Waiting for error group managing segments... Jul 19 04:45:23.583704815 (5h54m29.95628417s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.594296552 (5h54m29.966875907s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.598729564 (5h54m29.971308909s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.604912485 (5h54m29.97749184s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.605594740 (5h54m29.978174085s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.614360211 (5h54m29.986939566s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.617982884 (5h54m29.990562229s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.664577489 (5h54m30.037156844s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.675292701 (5h54m30.047872056s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.754290897 (5h54m30.126870252s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.017246894 (5h54m49.389826249s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.740402959 (5h54m50.112982314s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:50.623955253 (5h54m56.996534609s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:45:50.624007824 (5h54m56.996587169s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection Jul 19 04:45:50.624049063 (5h54m56.996628418s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection Jul 19 04:45:50.624147561 (5h54m56.996726916s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:34052->77.90.31.18:4806: use of closed network connection; context deadline exceeded Jul 19 04:45:50.624172359 (5h54m56.996751704s) [seg_pos={0 0}, upload=4698]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 04:45:50.854937698 (5h54m57.227517093s) [upload=4698]: Done uploading segments... Jul 19 04:45:50.854968076 (5h54m57.227547461s) [upload=4698]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.549343567s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 5.790713095s FAIL (2025-07-19 04:47:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ode:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection Jul 19 04:44:47.415361105 (5h53m53.78794046s) [seg_pos={0 0}, upload=4684]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 19 04:44:47.647064873 (5h53m54.019644228s) [upload=4684]: Done uploading segments... Jul 19 04:44:47.647091534 (5h53m54.019670889s) [upload=4684]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 19 04:45:21.280261101 (5h54m27.652840456s) [upload=4698]: Starting upload Jul 19 04:45:21.280319553 (5h54m27.652898908s) [upload=4698]: Done starting upload Jul 19 04:45:21.280540766 (5h54m27.653120121s) [upload=4698]: Uploading segments... Jul 19 04:45:21.280583858 (5h54m27.653163203s) [upload=4698]: Got next segment. Inline: false Jul 19 04:45:21.280620809 (5h54m27.653200164s) [upload=4698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:45:21.280625227 (5h54m27.653204583s) [upload=4698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:45:21.538947495 (5h54m27.91152685s) [seg_pos={0 0}, upload=4698]: Begin upload segment... Jul 19 04:45:21.539250875 (5h54m27.91183023s) [seg_pos={0 0}, upload=4698]: Done begin upload segment. Jul 19 04:45:21.539279179 (5h54m27.911858534s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540136549 (5h54m27.912715904s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540142360 (5h54m27.912721715s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540514533 (5h54m27.913093888s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540577964 (5h54m27.913157319s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540891614 (5h54m27.913470969s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540943022 (5h54m27.913522377s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541256162 (5h54m27.913835507s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541294124 (5h54m27.913873479s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541642441 (5h54m27.914221786s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541666407 (5h54m27.914245762s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542077954 (5h54m27.914657309s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542476085 (5h54m27.915055441s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543576250 (5h54m27.916155605s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543980244 (5h54m27.916559609s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.544371722 (5h54m27.916951078s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:23.300638583 (5h54m29.673217969s) [upload=4698]: Next returned nil segment Jul 19 04:45:23.300658502 (5h54m29.673237877s) [upload=4698]: Waiting for error group managing segments... Jul 19 04:45:23.583704815 (5h54m29.95628417s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.594296552 (5h54m29.966875907s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.598729564 (5h54m29.971308909s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.604912485 (5h54m29.97749184s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.605594740 (5h54m29.978174085s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.614360211 (5h54m29.986939566s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.617982884 (5h54m29.990562229s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.664577489 (5h54m30.037156844s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.675292701 (5h54m30.047872056s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.754290897 (5h54m30.126870252s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.017246894 (5h54m49.389826249s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.740402959 (5h54m50.112982314s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:50.623955253 (5h54m56.996534609s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:45:50.624007824 (5h54m56.996587169s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection Jul 19 04:45:50.624049063 (5h54m56.996628418s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection Jul 19 04:45:50.624147561 (5h54m56.996726916s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:34052->77.90.31.18:4806: use of closed network connection; context deadline exceeded Jul 19 04:45:50.624172359 (5h54m56.996751704s) [seg_pos={0 0}, upload=4698]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 04:45:50.854937698 (5h54m57.227517093s) [upload=4698]: Done uploading segments... Jul 19 04:45:50.854968076 (5h54m57.227547461s) [upload=4698]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.790739375s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 7.152747115s FAIL (2025-07-19 04:46:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ode:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection Jul 19 04:44:47.415361105 (5h53m53.78794046s) [seg_pos={0 0}, upload=4684]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 19 04:44:47.647064873 (5h53m54.019644228s) [upload=4684]: Done uploading segments... Jul 19 04:44:47.647091534 (5h53m54.019670889s) [upload=4684]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 19 04:45:21.280261101 (5h54m27.652840456s) [upload=4698]: Starting upload Jul 19 04:45:21.280319553 (5h54m27.652898908s) [upload=4698]: Done starting upload Jul 19 04:45:21.280540766 (5h54m27.653120121s) [upload=4698]: Uploading segments... Jul 19 04:45:21.280583858 (5h54m27.653163203s) [upload=4698]: Got next segment. Inline: false Jul 19 04:45:21.280620809 (5h54m27.653200164s) [upload=4698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:45:21.280625227 (5h54m27.653204583s) [upload=4698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:45:21.538947495 (5h54m27.91152685s) [seg_pos={0 0}, upload=4698]: Begin upload segment... Jul 19 04:45:21.539250875 (5h54m27.91183023s) [seg_pos={0 0}, upload=4698]: Done begin upload segment. Jul 19 04:45:21.539279179 (5h54m27.911858534s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540136549 (5h54m27.912715904s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540142360 (5h54m27.912721715s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540514533 (5h54m27.913093888s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540577964 (5h54m27.913157319s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540891614 (5h54m27.913470969s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540943022 (5h54m27.913522377s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541256162 (5h54m27.913835507s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541294124 (5h54m27.913873479s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541642441 (5h54m27.914221786s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541666407 (5h54m27.914245762s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542077954 (5h54m27.914657309s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542476085 (5h54m27.915055441s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543576250 (5h54m27.916155605s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543980244 (5h54m27.916559609s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.544371722 (5h54m27.916951078s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:23.300638583 (5h54m29.673217969s) [upload=4698]: Next returned nil segment Jul 19 04:45:23.300658502 (5h54m29.673237877s) [upload=4698]: Waiting for error group managing segments... Jul 19 04:45:23.583704815 (5h54m29.95628417s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.594296552 (5h54m29.966875907s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.598729564 (5h54m29.971308909s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.604912485 (5h54m29.97749184s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.605594740 (5h54m29.978174085s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.614360211 (5h54m29.986939566s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.617982884 (5h54m29.990562229s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.664577489 (5h54m30.037156844s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.675292701 (5h54m30.047872056s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.754290897 (5h54m30.126870252s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.017246894 (5h54m49.389826249s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.740402959 (5h54m50.112982314s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:50.623955253 (5h54m56.996534609s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:45:50.624007824 (5h54m56.996587169s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection Jul 19 04:45:50.624049063 (5h54m56.996628418s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection Jul 19 04:45:50.624147561 (5h54m56.996726916s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:34052->77.90.31.18:4806: use of closed network connection; context deadline exceeded Jul 19 04:45:50.624172359 (5h54m56.996751704s) [seg_pos={0 0}, upload=4698]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 04:45:50.854937698 (5h54m57.227517093s) [upload=4698]: Done uploading segments... Jul 19 04:45:50.854968076 (5h54m57.227547461s) [upload=4698]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.152762344s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.232702292s FAIL (2025-07-19 04:45:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== ode:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection Jul 19 04:44:47.415361105 (5h53m53.78794046s) [seg_pos={0 0}, upload=4684]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 19 04:44:47.647064873 (5h53m54.019644228s) [upload=4684]: Done uploading segments... Jul 19 04:44:47.647091534 (5h53m54.019670889s) [upload=4684]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 19 04:45:21.280261101 (5h54m27.652840456s) [upload=4698]: Starting upload Jul 19 04:45:21.280319553 (5h54m27.652898908s) [upload=4698]: Done starting upload Jul 19 04:45:21.280540766 (5h54m27.653120121s) [upload=4698]: Uploading segments... Jul 19 04:45:21.280583858 (5h54m27.653163203s) [upload=4698]: Got next segment. Inline: false Jul 19 04:45:21.280620809 (5h54m27.653200164s) [upload=4698]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 04:45:21.280625227 (5h54m27.653204583s) [upload=4698]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:45:21.538947495 (5h54m27.91152685s) [seg_pos={0 0}, upload=4698]: Begin upload segment... Jul 19 04:45:21.539250875 (5h54m27.91183023s) [seg_pos={0 0}, upload=4698]: Done begin upload segment. Jul 19 04:45:21.539279179 (5h54m27.911858534s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540136549 (5h54m27.912715904s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540142360 (5h54m27.912721715s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540514533 (5h54m27.913093888s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540577964 (5h54m27.913157319s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540891614 (5h54m27.913470969s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.540943022 (5h54m27.913522377s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541256162 (5h54m27.913835507s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541294124 (5h54m27.913873479s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541642441 (5h54m27.914221786s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.541666407 (5h54m27.914245762s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542077954 (5h54m27.914657309s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.542476085 (5h54m27.915055441s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543576250 (5h54m27.916155605s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.543980244 (5h54m27.916559609s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:21.544371722 (5h54m27.916951078s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Uploading piece... Jul 19 04:45:23.300638583 (5h54m29.673217969s) [upload=4698]: Next returned nil segment Jul 19 04:45:23.300658502 (5h54m29.673237877s) [upload=4698]: Waiting for error group managing segments... Jul 19 04:45:23.583704815 (5h54m29.95628417s) [noise=true , address=104.36.82.11:4114 , piece_id=REUPG2SPOHVX6LLCRWFVHDKGI2P5TLD3DE5LSYY5J6EAF2RVSNEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.594296552 (5h54m29.966875907s) [noise=true , address=104.36.82.13:4186 , piece_id=HX6XAMFNYTE6V7KZYTQOAZLN2YZXSTWFDKYR3627HFG7TAEJBVIQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.598729564 (5h54m29.971308909s) [noise=true , address=45.91.116.11:4118 , piece_id=KKOLBFMOOJRV6TU34PVEDJLKBSHJKFHHL6TTCAKPT5T3I2GIAQXA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.604912485 (5h54m29.97749184s) [noise=true , address=45.91.116.13:4155 , piece_id=Q3VJCUCFSYWLLTFD5UWJQ3QZVQW74TREPWPGRVWCG5P6BSDUXJLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.605594740 (5h54m29.978174085s) [noise=true , address=45.91.116.9:4263 , piece_id=O3WLYXI5NRCTNAMZDSBQLEZM4G44SE4KH2W6SONC3R7PSUGZXWYA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.614360211 (5h54m29.986939566s) [noise=true , address=45.91.116.7:4009 , piece_id=EKVJ2VE62WT4PRZUN2A7JVXJR7AYZCNZJLMU4FGZ3F46OEEEBR2A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.617982884 (5h54m29.990562229s) [noise=true , address=45.91.116.13:4153 , piece_id=BWIBS5VPAVMMERE4E6YRXH75E5B7WDHS3CSAA4RFJQYHROPFAAEQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.664577489 (5h54m30.037156844s) [noise=true , address=45.91.116.4:4939 , piece_id=3L7N4NEXA3UZSVKQCLS5UGTBJ6AKOSVTUYLVKLG5NMANC63Z6ZEA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.675292701 (5h54m30.047872056s) [noise=true , address=45.91.116.5:4299 , piece_id=6J3SQO5CTM6QV7AJV6H7FRBOPNN7PFEASWBCYVMNYL233TCA55LQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:23.754290897 (5h54m30.126870252s) [noise=true , address=104.36.82.2:4730 , piece_id=Y6VYNPQSU2BIQOOMZAPR7KGN5MC2ZIWJBT6A7N45BFRGE3O25ZRA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.017246894 (5h54m49.389826249s) [noise=true , address=77.90.31.19:4843 , piece_id=TXARH5UWBDAHWC6DD4F7WUXUJ75IDYC5SISHJNZNRFCP6ZPQZPNQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:43.740402959 (5h54m50.112982314s) [noise=true , address=77.90.31.21:4877 , piece_id=RCKZVX6PGYGLXYLHU4PIPPHIDYVBNTIB2IX34D4TULBIKRW36MLA, seg_pos={0 0}, upload=4698]: Done uploading piece. err: <nil> Jul 19 04:45:50.623955253 (5h54m56.996534609s) [noise=true , address=77.90.31.19:4842 , piece_id=VZW3T56NHLSXMCPCEIMRSEY37UYKYXMCFHNKLJVGWB5UQGKEPP5A, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: expected piece hash; context deadline exceeded; EOF Jul 19 04:45:50.624007824 (5h54m56.996587169s) [noise=true , address=77.90.31.12:4592 , piece_id=JFH3Z6VXDB6XOF3A7EXBHTNMLIPGBHKA24EPAJEA27LD6MDR447Q, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49604->77.90.31.12:4592: use of closed network connection Jul 19 04:45:50.624049063 (5h54m56.996628418s) [noise=true , address=77.90.31.13:4626 , piece_id=POIZZYWVZW7Z2TC56NHQIXR2YPPMPDMMCFC5Q3T6N4AKZEPXHEUQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53364->77.90.31.13:4626: use of closed network connection Jul 19 04:45:50.624147561 (5h54m56.996726916s) [noise=true , address=77.90.31.18:4806 , piece_id=6ZASNKDCF3ZXAG4QPJUCMTPLEBTSFGPHK3IIGKHHEXQSLI3HH7FQ, seg_pos={0 0}, upload=4698]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:34052->77.90.31.18:4806: use of closed network connection; context deadline exceeded Jul 19 04:45:50.624172359 (5h54m56.996751704s) [seg_pos={0 0}, upload=4698]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 19 04:45:50.854937698 (5h54m57.227517093s) [upload=4698]: Done uploading segments... Jul 19 04:45:50.854968076 (5h54m57.227547461s) [upload=4698]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.232729262s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.233052936s FAIL (2025-07-19 04:44:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== ectBegin Jul 19 04:44:18.065907349 (5h53m24.438486704s) [upload=4684]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 04:44:18.298815421 (5h53m24.671394776s) [seg_pos={0 0}, upload=4684]: Begin upload segment... Jul 19 04:44:18.298879134 (5h53m24.671458519s) [seg_pos={0 0}, upload=4684]: Done begin upload segment. Jul 19 04:44:18.298920913 (5h53m24.671500268s) [noise=true , address=77.90.31.21:4878 , piece_id=JHQO7CHERPMYBRN6BXKMIXAMITVRUG4IOJXNUXVJAV275JTRCIJA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.298937365 (5h53m24.67151671s) [noise=true , address=45.91.116.6:4974 , piece_id=OAQK5KBJSPCM2TL2OU6OZYKHDPFQNWQ3UW2L23O5WAS7CEK5OYXQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.299010965 (5h53m24.67159032s) [noise=true , address=104.36.82.14:4222 , piece_id=IQMSMRGPPTVOMRVTYCJVGCQ3QB6IAK443YM437AEC2T4X3WRKJVA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.299375053 (5h53m24.671954428s) [noise=true , address=104.36.82.4:4801 , piece_id=USAFHRX3SY2Z7FHJFAYJYQB7HMQJJZ5C6NIA47YY7ABBTVIWWKSA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.299477157 (5h53m24.672056512s) [noise=true , address=45.91.116.12:4226 , piece_id=NHP25ZCBDBZCA6MZISYKKDC5P46ILSUPUAB7UQ6YDSLMM4WYTE5A, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.299722587 (5h53m24.672301942s) [noise=true , address=45.91.116.10:4082 , piece_id=UN73W6XQTBN63VOY6QXUC5VGT25QSUUQAWVERWPHDGH3DUVTLMMA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.300038221 (5h53m24.672617576s) [noise=true , address=45.91.116.7:4010 , piece_id=I7AVYQCMTSIREKAFU7DZTQQ36IZRRI3ZQPXEFBNGGRSXJZOWVVYA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.300057948 (5h53m24.672637303s) [noise=true , address=77.90.31.21:4879 , piece_id=SKGUYZKJROPZAQJXZWSUIM5HKANSG7RYHIQ4S5XA6B5PKJ3OYNQQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.301427148 (5h53m24.674006503s) [noise=true , address=45.91.116.4:4938 , piece_id=SLZARMCTCAECHMUD6F7BDJWDZPT4PGDZ7KUAEMQEK5OBBBGXCV3Q, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.301746981 (5h53m24.674326336s) [noise=true , address=104.36.82.8:4972 , piece_id=FPALYDTSBT3CPNHASS54WAR7IOP22VZJUYHYJ2ZQBV5ODRND35MQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.302120575 (5h53m24.67469993s) [noise=true , address=77.90.31.19:4841 , piece_id=U23DFSMLZU4OZEN4R25W7OBECHMQWJZHNVSKEHH4OQYEW7KAKTFA, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.302388277 (5h53m24.674967632s) [noise=true , address=104.36.82.11:4113 , piece_id=VLYK6K6VKJETKRSIYOP2OMBRUMIVIDVT74FKZMI32UBTQLK32J5A, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.302780678 (5h53m24.675360043s) [noise=true , address=104.36.82.3:4765 , piece_id=HAIQIGTPO5WQV5R5SKDTBC46LABN6R73AXM4NN4O7E5PWK3II2KQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.302992684 (5h53m24.675572039s) [noise=true , address=77.90.31.17:4771 , piece_id=R2QMNRA6MEUWLCO5KFSVXSOSNW5LYH6HIKPGARXMOOIR5JMW5U5A, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.303508621 (5h53m24.676087976s) [noise=true , address=77.90.31.11:4554 , piece_id=NSSHVABSHVN5I6H5FCFPXQE66XYKXH55RJDLW47663SGVXA7NGFQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:18.304216315 (5h53m24.67679567s) [noise=true , address=77.90.31.19:4843 , piece_id=RXJFNY4LORYZHTEBN2WJ73BJVODUWTJF5JC5EBPA5WXV6BGTO5GQ, seg_pos={0 0}, upload=4684]: Uploading piece... Jul 19 04:44:20.121983916 (5h53m26.494563271s) [upload=4684]: Next returned nil segment Jul 19 04:44:20.122001110 (5h53m26.494580465s) [upload=4684]: Waiting for error group managing segments... Jul 19 04:44:20.427050681 (5h53m26.799630036s) [noise=true , address=45.91.116.4:4938 , piece_id=SLZARMCTCAECHMUD6F7BDJWDZPT4PGDZ7KUAEMQEK5OBBBGXCV3Q, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.429966960 (5h53m26.802546315s) [noise=true , address=45.91.116.12:4226 , piece_id=NHP25ZCBDBZCA6MZISYKKDC5P46ILSUPUAB7UQ6YDSLMM4WYTE5A, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.440143962 (5h53m26.812723317s) [noise=true , address=104.36.82.4:4801 , piece_id=USAFHRX3SY2Z7FHJFAYJYQB7HMQJJZ5C6NIA47YY7ABBTVIWWKSA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.458511514 (5h53m26.83109087s) [noise=true , address=45.91.116.10:4082 , piece_id=UN73W6XQTBN63VOY6QXUC5VGT25QSUUQAWVERWPHDGH3DUVTLMMA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.486054523 (5h53m26.858633878s) [noise=true , address=104.36.82.3:4765 , piece_id=HAIQIGTPO5WQV5R5SKDTBC46LABN6R73AXM4NN4O7E5PWK3II2KQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.489120058 (5h53m26.861699413s) [noise=true , address=104.36.82.8:4972 , piece_id=FPALYDTSBT3CPNHASS54WAR7IOP22VZJUYHYJ2ZQBV5ODRND35MQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.513036868 (5h53m26.885616213s) [noise=true , address=104.36.82.11:4113 , piece_id=VLYK6K6VKJETKRSIYOP2OMBRUMIVIDVT74FKZMI32UBTQLK32J5A, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.517197138 (5h53m26.889776493s) [noise=true , address=45.91.116.7:4010 , piece_id=I7AVYQCMTSIREKAFU7DZTQQ36IZRRI3ZQPXEFBNGGRSXJZOWVVYA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.534307075 (5h53m26.90688643s) [noise=true , address=45.91.116.6:4974 , piece_id=OAQK5KBJSPCM2TL2OU6OZYKHDPFQNWQ3UW2L23O5WAS7CEK5OYXQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:20.643866654 (5h53m27.016446009s) [noise=true , address=104.36.82.14:4222 , piece_id=IQMSMRGPPTVOMRVTYCJVGCQ3QB6IAK443YM437AEC2T4X3WRKJVA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: <nil> Jul 19 04:44:47.415116356 (5h53m53.787695711s) [noise=true , address=77.90.31.17:4771 , piece_id=R2QMNRA6MEUWLCO5KFSVXSOSNW5LYH6HIKPGARXMOOIR5JMW5U5A, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:48080->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:48080->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48080->77.90.31.17:4771: use of closed network connection Jul 19 04:44:47.415170961 (5h53m53.787750316s) [noise=true , address=77.90.31.11:4554 , piece_id=NSSHVABSHVN5I6H5FCFPXQE66XYKXH55RJDLW47663SGVXA7NGFQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:42028->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:42028->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42028->77.90.31.11:4554: use of closed network connection Jul 19 04:44:47.415215326 (5h53m53.787794681s) [noise=true , address=77.90.31.21:4878 , piece_id=JHQO7CHERPMYBRN6BXKMIXAMITVRUG4IOJXNUXVJAV275JTRCIJA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:54818->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:54818->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54818->77.90.31.21:4878: use of closed network connection Jul 19 04:44:47.415267185 (5h53m53.787846541s) [noise=true , address=77.90.31.19:4841 , piece_id=U23DFSMLZU4OZEN4R25W7OBECHMQWJZHNVSKEHH4OQYEW7KAKTFA, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:48100->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:48100->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48100->77.90.31.19:4841: use of closed network connection Jul 19 04:44:47.415311831 (5h53m53.787891186s) [noise=true , address=77.90.31.21:4879 , piece_id=SKGUYZKJROPZAQJXZWSUIM5HKANSG7RYHIQ4S5XA6B5PKJ3OYNQQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: write tcp 188.166.225.170:56502->77.90.31.21:4879: use of closed network connection; write tcp 188.166.225.170:56502->77.90.31.21:4879: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56502->77.90.31.21:4879: use of closed network connection Jul 19 04:44:47.415347489 (5h53m53.787926844s) [noise=true , address=77.90.31.19:4843 , piece_id=RXJFNY4LORYZHTEBN2WJ73BJVODUWTJF5JC5EBPA5WXV6BGTO5GQ, seg_pos={0 0}, upload=4684]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40322->77.90.31.19:4843: use of closed network connection Jul 19 04:44:47.415361105 (5h53m53.78794046s) [seg_pos={0 0}, upload=4684]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 19 04:44:47.647064873 (5h53m54.019644228s) [upload=4684]: Done uploading segments... Jul 19 04:44:47.647091534 (5h53m54.019670889s) [upload=4684]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 30.233073405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 14.332247926s FAIL (2025-07-19 04:43:39 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 14.332279086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 20.487983271s FAIL (2025-07-19 04:42:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 20.48800879s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.980389903s FAIL (2025-07-19 04:41:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.980415212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.745871299s FAIL (2025-07-19 04:40:58 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.745897749s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.766103831s FAIL (2025-07-19 04:40:01 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.766135913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.876884225s FAIL (2025-07-19 04:39:38 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.876941163s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 8.172474146s FAIL (2025-07-19 04:38:43 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.172500405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 7.936620148s FAIL (2025-07-19 04:38:01 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.936650837s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 8.517237563s FAIL (2025-07-19 04:36:53 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.517258522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.770395492s FAIL (2025-07-19 04:35:58 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.77041504s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.785533014s FAIL (2025-07-19 04:35:25 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.785555007s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 8.749265901s FAIL (2025-07-19 04:34:15 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.7492866s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.889502271s FAIL (2025-07-19 04:33:16 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.889527019s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 7.682713762s FAIL (2025-07-19 04:32:14 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.682734401s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 7.794335646s FAIL (2025-07-19 04:31:59 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.794356145s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.46187637s FAIL (2025-07-19 04:31:01 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.462035705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 7.3889966s FAIL (2025-07-19 04:30:42 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.389018562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.713837259s FAIL (2025-07-19 04:29:28 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.713857627s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.831437064s FAIL (2025-07-19 04:28:21 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.831455408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.871309245s FAIL (2025-07-19 04:27:20 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.871339763s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 7.213000188s FAIL (2025-07-19 04:26:15 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.213018584s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.921282922s FAIL (2025-07-19 04:25:27 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.921305565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.879116074s FAIL (2025-07-19 04:24:27 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.879177162s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 7.646039595s FAIL (2025-07-19 04:24:12 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.646065063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.783542902s FAIL (2025-07-19 04:23:17 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.783571387s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.687630634s FAIL (2025-07-19 04:22:23 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.687654761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 8.002906691s FAIL (2025-07-19 04:21:51 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.002930628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 7.431138438s FAIL (2025-07-19 04:21:26 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.431164469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 8.427496381s FAIL (2025-07-19 04:20:18 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.427518775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 11.429936784s FAIL (2025-07-19 04:19:18 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 11.429964416s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 8.390492672s FAIL (2025-07-19 04:18:01 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 8.390518592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.436290144s FAIL (2025-07-19 04:17:11 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.436321072s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.884124883s FAIL (2025-07-19 04:16:45 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.884150313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.854864966s FAIL (2025-07-19 04:15:50 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.854891155s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 15.387373842s FAIL (2025-07-19 04:15:08 UTC) orbiter warning: ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== d network connection Jul 19 03:55:50.588700290 (5h4m56.961279635s) [noise=true , address=77.90.31.18:4805 , piece_id=5UNY5TE5BUPQMBIF72FDWT4YVDWGEZZ6LYVBO7ANKDKBHJPXAKIA, seg_pos={0 0}, upload=4041]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Jul 19 03:55:50.588752119 (5h4m56.961331474s) [seg_pos={0 0}, upload=4041]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:55:50.588840779 (5h4m56.961420134s) [upload=4041]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:55:50.588846049 (5h4m56.961425404s) [upload=4041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:55:50.862788155 (5h4m57.23536751s) [upload=4041]: Done uploading segments... Jul 19 03:55:50.862808112 (5h4m57.235387467s) [upload=4041]: Upload finished. err: <nil> Jul 19 03:57:00.952603526 (5h6m7.325182881s) [upload=4057]: Starting upload Jul 19 03:57:00.952662148 (5h6m7.325241523s) [upload=4057]: Done starting upload Jul 19 03:57:00.952755756 (5h6m7.325335111s) [upload=4057]: Uploading segments... Jul 19 03:57:00.952878392 (5h6m7.325457747s) [upload=4057]: Got next segment. Inline: false Jul 19 03:57:00.952914330 (5h6m7.325493675s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 03:57:00.952919900 (5h6m7.325499255s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 03:57:01.188368206 (5h6m7.560947561s) [seg_pos={0 0}, upload=4057]: Begin upload segment... Jul 19 03:57:01.188405837 (5h6m7.560985192s) [seg_pos={0 0}, upload=4057]: Done begin upload segment. Jul 19 03:57:01.188433881 (5h6m7.561013236s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188471683 (5h6m7.561051098s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.188793689 (5h6m7.561373044s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189154039 (5h6m7.561733394s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189428814 (5h6m7.562008169s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189450466 (5h6m7.562029821s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189780508 (5h6m7.562359863s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.189821466 (5h6m7.562400821s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190223174 (5h6m7.562802529s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190313167 (5h6m7.562892512s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190468354 (5h6m7.563047709s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190520664 (5h6m7.563100019s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190731417 (5h6m7.563310782s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.190896333 (5h6m7.563475698s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191142764 (5h6m7.563722119s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:01.191246532 (5h6m7.563825887s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Uploading piece... Jul 19 03:57:02.948067971 (5h6m9.320647326s) [upload=4057]: Next returned nil segment Jul 19 03:57:02.948086357 (5h6m9.320665712s) [upload=4057]: Waiting for error group managing segments... Jul 19 03:57:03.238444561 (5h6m9.611023916s) [noise=true , address=45.91.116.14:4189 , piece_id=35KEJB2BISQK6H3NS5JYTIU4RLH2ZJYL3IDIREW2UKZ55TUOW6GA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.250389355 (5h6m9.6229687s) [noise=true , address=45.91.116.4:4937 , piece_id=B64BEYMBTSPGDBGPHDVNGMZMFJZBMFF7DM4JSTS5SOS7GOATTPDA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.251034119 (5h6m9.623613474s) [noise=true , address=45.91.116.6:4973 , piece_id=6H5JF44FJG2TPRR7EBUBRGEKK4RDZY3LDEEM43X43R4MELZ5I53Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.263366183 (5h6m9.635945558s) [noise=true , address=45.91.116.10:4081 , piece_id=TTGDQEXMTXEF3FFBNGM3OORNLI7UJLP425HINTJ6OQIVRQG5ZQRA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.321950553 (5h6m9.694529908s) [noise=true , address=104.36.82.11:4115 , piece_id=UXSVWNLCG2TRR6GA5RIASW2HVEHJHVGEJOTJX57QQZ76GWR7D3FQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.332673900 (5h6m9.705253255s) [noise=true , address=104.36.82.3:4765 , piece_id=ISHXBHLWCHKBWVMOQXNO2SKNY6KTZIIN6OXMVSHJA6VEZ7SE4X6Q, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.341648953 (5h6m9.714228308s) [noise=true , address=45.91.116.13:4155 , piece_id=L7R7FOT3ES52GLCQQPIW2HWMJP23LD4333OFHIWI5UNTT47KCTSQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.343719043 (5h6m9.716298398s) [noise=true , address=45.91.116.8:4048 , piece_id=JT72V4S4RFG2FWUFTIP2D5K5QL5ZVNAYH42JV3TAR2O4STVYCLAQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.347782328 (5h6m9.720361683s) [noise=true , address=104.36.82.15:4258 , piece_id=LZS53AW4U6X4R6V7LCP6WV2VY4BEMUFEDAYFDO5EPCAVLASZCEPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.354893633 (5h6m9.727472998s) [noise=true , address=45.91.116.9:4263 , piece_id=AWSRDUCWXEXSDS5YBPROGCRJPYJH4G3T4JELM3JXF4ULWBUNXAZA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:03.380020720 (5h6m9.752600075s) [noise=true , address=104.36.82.12:4150 , piece_id=HRXGHVIFPHWKOPIPWZOIRFXS6XV6CFWG25WQKVLOUQVNTNJHV2HA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.220478869 (5h6m11.593058225s) [noise=true , address=77.90.31.11:4555 , piece_id=JVPDEWNIZDRUKT5PAPL3C6JCBYJC5INX7HUGGBI2PGI64P6AZ4AA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275708140 (5h6m11.648287495s) [noise=true , address=77.90.31.15:4699 , piece_id=ZROX6BVXQCZ6EB6AMB6OESSZD3QG2H6HIQDVKJGWCJMB6QR62TWQ, seg_pos={0 0}, upload=4057]: Done uploading piece. err: <nil> Jul 19 03:57:05.275729681 (5h6m11.648309036s) [seg_pos={0 0}, upload=4057]: Segment reached optimal threshold of 13 pieces. Jul 19 03:57:05.275878185 (5h6m11.64845753s) [noise=true , address=77.90.31.21:4877 , piece_id=VIGIC56R7QAURK3C34ATOF5H467ZWSBTKK6ZCK3IXZYQGHMJTD7A, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.275971504 (5h6m11.648550849s) [noise=true , address=77.90.31.19:4842 , piece_id=ZR6JTOMFIQOO3DKVMYAM54CS6TDHHCMN2Y2P6RLSP4BWK5MAHTPA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Jul 19 03:57:05.276081555 (5h6m11.64866091s) [noise=true , address=77.90.31.10:4519 , piece_id=YHGR7JZKLUC4KIQVGZPK4TECJ6KGADCEB3OCM42223C5LU6J5JVA, seg_pos={0 0}, upload=4057]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34236->77.90.31.10:4519: use of closed network connection Jul 19 03:57:05.276132543 (5h6m11.648711898s) [seg_pos={0 0}, upload=4057]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 03:57:05.276207365 (5h6m11.64878672s) [upload=4057]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 03:57:05.276211424 (5h6m11.648790769s) [upload=4057]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 03:59:23.699026037 (5h8m30.071605382s) [upload=4057]: Done uploading segments... Jul 19 03:59:23.699046667 (5h8m30.071626022s) [upload=4057]: Upload finished. err: metaclient: context deadline exceeded Duration: 15.387396394s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)