Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
5.841453199s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:30:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.841477124s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.882498935s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:29:59 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.882530075s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
5.804972536s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:28:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.804999307s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.878691627s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:28:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.87871987s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.80167248s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:27:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.801705303s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.723112749s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:26:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.723132839s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.859557858s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:25:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.859591662s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.877801488s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:24:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.877825223s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.783821072s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:24:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.783844536s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.928343545s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:23:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.928359578s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.87456533s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:22:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.874591561s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.772463408s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:21:44 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.772476904s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.847094924s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:20:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.847119421s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.855811883s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:19:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.855830781s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.774654241s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:18:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.774678928s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.798628566s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:18:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.798661959s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.829991059s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:16:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.830019945s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.861603913s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:16:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.861635835s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.778955316s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:15:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.778981767s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
7.309026087s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:14:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.309052527s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.82608315s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:13:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.826115111s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
7.593976359s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:12:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.593995806s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
6.855659043s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:12:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.855680025s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
7.593368609s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:11:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.593408916s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
7.808529207s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:11:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.808571257s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
7.557112578s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:11:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.557142096s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.803626779s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:10:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.803678959s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
6.273889168s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:09:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.273916441s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.723626366s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:09:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.723656955s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
6.258837703s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:08:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.258863765s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.800446821s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:07:28 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.800472251s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
6.243362348s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:07:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.243401644s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.889836125s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:06:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.889866374s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.79181377s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:05:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.791839469s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.131117246s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:04:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.131131966s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.032055988s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:03:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.032083612s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.800452031s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:02:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.800481828s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
7.623000628s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:02:22 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 7.623051987s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.780255285s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:01:36 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.780281747s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.751009933s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:01:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.751040292s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
8.922899171s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:00:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 8.922924108s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.68653278s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 13:00:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.686574981s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.80367283s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:59:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.803690975s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.805526418s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:58:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.805551697s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.823399037s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:56:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 5.823432332s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
6.520573961s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:55:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 6.520597426s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
10.938225637s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:55:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 10.93827429s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.235842845s trace:
https://jaeger.dev.storj.io/trace/3df87cd1f6b5da4b
FAIL (2026-04-07 12:54:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 13 but got 0)
======LOGS======
, piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.927993841 (769h48m59.840331472s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928219727 (769h48m59.840557358s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.928646722 (769h48m59.840984363s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929137830 (769h48m59.841475461s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.929822601 (769h48m59.842160232s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930245677 (769h48m59.842583308s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930583359 (769h48m59.84292099s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.930911883 (769h48m59.843249514s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:16.932027616 (769h48m59.844365247s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Uploading piece...
Apr 7 12:54:46.019289340 (769h49m28.931626981s) [upload=738699]: Next segment err: context deadline exceeded
Apr 7 12:54:46.019398220 (769h49m28.931735851s) [noise=true , address=145.79.227.5:4227 , piece_id=JWT5TFITRVWTIQSRMJURTKX7I5XWIQNVV27OBMHQ676XCNYXDZ5A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: read tcp 188.166.225.170:43248->145.79.227.5:4227: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019453186 (769h49m28.931790817s) [noise=true , address=145.79.227.51:4413 , piece_id=IUWT2MACH3HBOQHA4T3QERKPIGV2VVFMTDAGN6T2DIJXOQZWOOAQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1oMVt1KZoNSDX5jKHBXw4WSr69eQYxcY3Z1nxLBsHr19Btw7Gy, address:145.79.227.51:4413): protocol: read tcp 188.166.225.170:52598->145.79.227.51:4413: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019597444 (769h49m28.931935075s) [noise=true , address=145.79.227.4:4188 , piece_id=RVYK3JXEUTIP334ZM745SW6MHIBA2WJ2EFHZTXH6PKHX6JJJONKQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12pLFHQ2Xj6n53w9eLQtMB4A29k3Xh6w716z5s2CGxwPnBoZcgZ, address:145.79.227.4:4188): protocol: read tcp 188.166.225.170:33396->145.79.227.4:4188: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019621000 (769h49m28.931958631s) [noise=true , address=145.79.227.52:4448 , piece_id=EICELYVLGW4WXDFARAFUNHY26Q7E3CZDQELGGK46KUY76OOJE5JA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12JfhuZrWmNr3sBwxV9DKCs3CNQmeGEYnNN1PLwNtAE1dN4J77i, address:145.79.227.52:4448): protocol: read tcp 188.166.225.170:45224->145.79.227.52:4448: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019719180 (769h49m28.932056811s) [noise=true , address=145.79.227.46:4604 , piece_id=PUXECO7O5WZZVZC5T5MB6NPOQ2LFE7727RNRNZJ5OXPQTVWKE6CA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:129d1v3mjqB3FHC2VmbY9Sih6C5fRDrV3kGLiv9NfkDc7JovXjy, address:145.79.227.46:4604): protocol: read tcp 188.166.225.170:39080->145.79.227.46:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019744769 (769h49m28.93208239s) [noise=true , address=145.79.227.47:4268 , piece_id=O5S4RZDEV7WUI4A7II3ADJTPV7SP3KSBVUJWDTBIJCYLR6LO3WDA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:165gcpfWaKHcMWugoiB5cTK8hpFYp1iymG8xnBdLgPuA2o1wkj, address:145.79.227.47:4268): protocol: read tcp 188.166.225.170:56536->145.79.227.47:4268: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019845153 (769h49m28.932182784s) [noise=true , address=145.79.227.22:4514 , piece_id=RIFON4MRXKKBKQKVBW4QY4VV7HJL6UA6YHEYOJ7UUC7O3ZKTBNTA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Nh9nHAnKWgu1ggCAZ7WcUadbvsPqAgbqzGrBnSaDMqDhquQ4R, address:145.79.227.22:4514): protocol: read tcp 188.166.225.170:60470->145.79.227.22:4514: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.019962960 (769h49m28.932300591s) [noise=true , address=145.79.227.24:4584 , piece_id=WVISTF5A4AAYGKQ74H7MM5J5MNREFVRFIPEQFLJKSFLT7LZZDR4A, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12TMMXMjWNxBgccLQMbuExNxgwvzQpNH4cizjaPZDufcjvTqdeh, address:145.79.227.24:4584): protocol: read tcp 188.166.225.170:51274->145.79.227.24:4584: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020012215 (769h49m28.932349846s) [noise=true , address=145.79.227.16:4915 , piece_id=KSTWAKJFA2JRBLHIVZR7EUMFFSTOIYVLZ2EQEZ72NDXVDUBH5NLA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12fqZipCJaNd9u5TvdWZ9nJFs26MiGtYX3Bbh41eQWP1Z4eJRBt, address:145.79.227.16:4915): protocol: read tcp 188.166.225.170:45650->145.79.227.16:4915: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020080728 (769h49m28.932418359s) [noise=true , address=145.79.227.48:4304 , piece_id=C66UYBFCO5TNHN4W5CPYXMH62HGLPAUIZDIWKPKJNNFXZ2E7FUNA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12aK78a6w6pX4fS7DTuJ7uHHvQfPvQjNaVVGgFc2uLGyDzbNXGG, address:145.79.227.48:4304): protocol: read tcp 188.166.225.170:47112->145.79.227.48:4304: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020133970 (769h49m28.932471601s) [noise=true , address=145.79.227.12:4768 , piece_id=5FAY36KQC3XG64TT65RKN226HUYYSFDB7KHN5E7STIPTWMTPN53Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hALeWp4XQEXcmWxXbWU3TTbyLdEF9Pn2h29WwtV5T5jPs3Esq, address:145.79.227.12:4768): protocol: read tcp 188.166.225.170:43972->145.79.227.12:4768: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020198795 (769h49m28.932536426s) [noise=true , address=145.79.227.44:4198 , piece_id=LV4RUPHOZVE5C4EFRMA7HJM5DPGBEOEPNK4GLK6RBGJP3O5SCVSQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL, address:145.79.227.44:4198): protocol: read tcp 188.166.225.170:38322->145.79.227.44:4198: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020265384 (769h49m28.932603065s) [noise=true , address=145.79.227.31:4797 , piece_id=X4QQFGGP5USDLYAUE6OW7RRLLZ4YMA6XFMPEJQEDNMF2W3IEHR4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:1VwwhXQqXCHcgP37bcQAUA6urv5YGVt7NCpRNrqMKSSE5AScK9, address:145.79.227.31:4797): protocol: read tcp 188.166.225.170:44742->145.79.227.31:4797: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020480248 (769h49m28.932817879s) [noise=true , address=145.79.227.15:4874 , piece_id=V2MLU2WJSKFU5OBH3NRHUWFX2GHTGXE5WVESZ4HF3UA7CRA4AS4Q, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:52732->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020598817 (769h49m28.932936448s) [noise=true , address=145.79.227.17:4971 , piece_id=UZMLA2C4VCVVPD5HLUYJVNZA36M7ZFA6DJ2OZI4ENEEF2BJYKTYA, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:53254->145.79.227.17:4971: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020710312 (769h49m28.933047944s) [noise=true , address=145.79.227.43:4162 , piece_id=TWA66BEG4MSFC4BJ54PXJX7PRAXADRPFOO3WU2ZBKCQD26BNL5QQ, seg_pos={0 0}, upload=738699]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:145.79.227.43:4162): protocol: read tcp 188.166.225.170:53574->145.79.227.43:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 7 12:54:46.020732104 (769h49m28.933069735s) [seg_pos={0 0}, upload=738699]: Done waiting for segment. successful: 0 optimal: 13 errs: <nil>
Duration: 30.235862645s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
3.769930304s trace:
https://jaeger.dev.storj.io/trace/52aa99243c8e0904
FAIL (2026-04-07 10:36:34 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.769962256s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
3.772277243s trace:
https://jaeger.dev.storj.io/trace/52aa99243c8e0904
FAIL (2026-04-07 10:35:43 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.772302843s
core
uplink
us-select-1
us1-placement-20
|